Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Lord Dudeguy
Sep 17, 2006
[Insert good English here]
A CFO came in...

:v: We're expanding branches and employees by 20%.
:cool: Ok we'll need to order more storage/memory/servers/etc to meet the increased demand.
:v: You don't have any room in your budget. Request denied.

Adbot
ADBOT LOVES YOU

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

psydude posted:

Move him over to the normal storage and exchange servers and watch your budget magically increase.

Two words: Storage Quotas

Today's going to be interesting.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

skipdogg posted:

Happens every year when something like this comes out.

http://www.globalknowledge.com/training/generic.asp?pageid=3158&country=United+States

People see dollar signs, chase a cert, get the cert and hope to cash in. Like you said, a couple years ago it was the CISSP, then the VCP, now the PMP.

That poo poo has got to be in California-bucks. I'm in Southeast New England and you're lucky to see a VCP get $70,000 around here.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

GreenNight posted:

He says it will remove face to face and phone conversations and also be non productive.

"Lync is face to face and phone conversations." :smug:

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Siochain posted:

Poor VoIP quality while tethered to iPhone while camping in middle of nowhere somewhere in Montana or Colorado or something. I'm...shocked? That it worked at all, that is.
woooooo Tuesdays!

Wait, they're using their phone data to use phone voice? They couldn't just dial the number?

Lord Dudeguy
Sep 17, 2006
[Insert good English here]
So the merger finished last weekend. 6 remote sites, 5 servers, 100GB of e-mail merged in, file services, etc... I'm finding it difficult to come down from the rage/stress... probably because:

- Acquired company employees were local shared-account admins on Intel i7 workstations and we're putting them on VDI so "WHY IS IT SO SLOOOOW". Oh, and they're required to remember passwords now.
- Now that the merger was "successful" (firefighting notwithstanding) all of a sudden all put-off large projects have a due-date of "Today" because "Well the merger is OVER so you're not doing anything".
- Co-worker managed to take down the entire VDI cluster while adding a host to it.
- Boss is lord of Scope Creep, so what was "You get a desktop and e-mail access only", turned into "And SQL access, and software access, and a pony..." and so forth.

I mentioned before that we were given 18 hours to cut everything over. That didn't happen. We were actually given 9 hours, because departments started shuffling in first thing on the day we were given to flip everything. We "cheated" and let ourselves into the remote locations at end-of-business. The lawyers were not happy.

I've been going flat-out since 9/2. Very little sleep. No breaks. No comp time. Two days off isn't going to help me get down from this stress-pile I'm sitting on.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]
Bunch of servers go down. Co-worker comes out of the DC:

"I was tracing some network cables and unplugged the iSCSI switch."

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

stevewm posted:

The local bank I financed my mortgage through uses one of these.. I don't get the point of it either, well at least how that particular institution implements it anyways..

They used it to send me empty loan documents that had zero personal or confidential information on them whatsoever. It was a major pain in the rear end registering for the thing as they had some obscenely complicated password requirements setup on it. Once I filled the forms out, I attached and sent them back like I would any other attachment...

Hi, it's me! The SysAdmin for your local bank!

Our SOX auditors/consultants have senior management so afraid of their own shadows, that it's become easier to simply make it impossible to do any work, rather than risk leaking any data.

Our original email flow worked like this:

Any e-mail with *keyword* in the subject would be sent to a secure mail appliance, which forced customers to set up (and manage) their own account/passwords, with expiration and complexity requirements. The appliance also stripped out any HTML/links from e-mails anyway. Any e-mail with *pattern* in the body would be sent to secure mail. Any e-mail from *Application* would be sent to secure mail.

In effect, people were getting stuff crammed in that horrible interface time and again for stuff that wasn't even sensitive. Is your phone number written 555-555-5555 in the body of the message? OFF TO SECURE MAIL WITH YOU because it looks like a Canadian social insurance number (xxx-xxx-xxx). Sending a large ZIP file of house pictures? SECURE MAIL. E-mail about "Yoga Class Safety Measures"? SECURE MAIL.

I just started an alternative mail flow that allows us to whitelist domains, provided that those domains utilize TLS. We then TLS-require those domains, so if there's no wire encryption, the message bounces.

I seriously hate "Email Security" appliances with a passion, so whatever it takes to take them out of the loop (and still be in compliance), I'm all for.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

stevewm posted:

I just thought it was so asinine... They send me empty documents with zero information that would even need to be secured.

If the user used a keyword, that's a training issue.

quote:

But requested I just scan the filled out documents (now with every single bit of my personal information on them) and send them back via regular mail. What exactly was the point of using the "secure" thing in the first place?!?!

If you (the customer) send me (the institution) a sensitive document, it's not my responsibility if it gets leaked. If I (the institution) send you (the customer) a sensitive document, it's on me if it gets leaked (until it reaches your server, then it's on you).

quote:

They obviously have trouble with customer's not understanding this system too.. As the email notifying me I had a secure message was immediately followed by a cut and pasted email from the same person several pages long with detailed step by step directions for registering, logging in, and using the "secure" email website.

The folly of e-mail security appliance deployment is that companies like to blanket-deploy it. Instead of intelligently handling e-mail if the recipient does/does not support TLS encryption, they just say "gently caress it" and throw everything at the appliance.

That's really all those appliances are for: If a recipient server only supports unencrypted SMTP, you use secure mail to force the recipient to retrieve their messages over HTTPS. The message never gets "transmitted" unless the wire is encrypted. Or, you set up TLS whitelists to avoid the whole mess.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]
:edit: Never mind. Cranky as hell.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]
loving Windows DFS Replication services and its voodoo.

We had a ESX physical host croak last week, and it took a DFS node with it. When the node came back all logs and services reported that they were A-OK, so we didn't think anything of it. Even did a couple test file reps.

Today it all goes to hell. Reports came in that files weren't replicating from node to node. We take a look and sure enough, DFSR is hung. Restart the service and the world comes to an end. Deleted files are coming back, new files are being deleted, file modifications are reverting back to previous state. We do daily backups, so a great deal of work done today is gone.

Meanwhile, DFS status says everything's 100% replicated, even while files are getting blown away in droves. Event logs say everything's OK. Health report says everything's OK.

The replication catchup still hasn't completed yet. I shudder to think what things will look like in the morning.

Of all the services, DFS is this crazy Pandora's box that Microsoft doesn't want you to know about. I seriously hate this role.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Agrikk posted:

As you've discovered, you have to be really, really careful about reintroducing a down node back into a DFS namespace.

So how the heck do you do it? Do you basically have to pre-emptively robocopy the entire volume again before re-introducing the node? If that's the case, then I'd need to set DFSR to manual service startup and not automatic, which is the default.

:edit: ^^^ Dear christ. I'm going to start shopping for a better method. poo poo, SAMBA w/ rsync has better failure recovery features than that.

Lord Dudeguy fucked around with this message at 01:34 on Nov 13, 2013

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Caged posted:

Sorry to poo poo the thread up but I might as well follow this one through, can you kick the node out the namespace from the hosts that are still up, or do you have to boot it up with the network disabled?

You can punt a node from any online node.

You can even get the DFS admin panel on any server on the domain with a Domain Admin logged in.

DFS is largely defined in Active Directory. If the node came online unexpectedly, the other nodes won't talk to it because they already pulled the latest DFS config. Eventually, the downed node will catch up and stop talking.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

deimos posted:

Regarding DFS: How the gently caress do 3 nodes losing 1 get split-brain?

That's what I want to loving know.

Fortunately the re-sync of the down node restored a good portion of data once it finished (overnight). Almost like it was replaying transactional logs as it went from volume to volume.

For those who still had missing files, I was able to dive into the share's DFSRPrivate folder and restore the... what are they, snapshots? Fragments? of the files.

We're trying to establish some sort of way to psychically establish when to consider a DFS node "down" and pull it out of DFSR and Namespace. It's 100% judgement call. If the server's up, says it's up, and the logs are clean, it might still be "down", but HEY WHO KNOWS, RIGHT?

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

deimos posted:

Not sure if sarcastic or just sleep deprived. Either way :bravo:.

Sarcasm. In our scenario:

The node went down until it failed over in VMware to a new host.

The node came back up and said it was replicating. The health reports were good, logs were good, and we tested a few shares to make sure that they were still replicating to the other nodes.

The DFSR service died on the node that bounced, and reports trickled in that the other nodes weren't receiving files. Health reports were still good (all three nodes), logs were good (all three), but files wouldn't sync.

Restarted DFSR on the bounced node, all hell breaks loose.

Now, it was that node that was the troublesome one, but with all signs originally pointing to A-OK post-failure, what's an Admin to do? We'd have to had known that this problem would have happened a week later, before we decided to remove the node from the cluster.

Ok, maybe we should remove a node every time they reboot? Seriously? Is DFS that fragile?

Ok, maybe we should remove a node every time it acts funny. Then why do the health reports and logs say everything's fine? Is DFS reporting/logging that awful?

Ok, let's assume that DFS is that fragile, and the reporting/logging is that awful. Then why hasn't Redmond been burned to a cinder yet over such a crap-rear end product that's been a feature of Windows NT/20xx for over a decade?

:edit:

At this point our operational stance is do not trust VMWare failover. If an ESX host dies during production, all affected servers will be rebooted immediately after HA services say the host is back online.

That's no way to run a datacenter, but I'm honestly at a loss as to what else we can do - shy of dumping DFS altogether.

Lord Dudeguy fucked around with this message at 03:05 on Nov 14, 2013

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

KennyTheFish posted:

While I understand your WTF at DFS. I am wondering why you are running high available clustering (DFS) on high available clustering (VMware failover)? I don't do datacentre level stuff, so I could just be missing something.

VMWare failover is for on-site host failover.

DFS is for site-to-site failover.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]
Welp, DFS hosed me again.

Trying to replicate User folders, started replicating AppData on one user's folder. Crashed.

Removed node. Waited for config to rep. Re-added node with a "good" node as master. All hell is breaking loose again.

Thank god for "DFSRPrivate/Conflict and Deleted". I'd be fired without it.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Vin BioEthanol posted:

I'm not actually involved in this but a manager here has somehow successfully talked vdi admins into + gotten a change approved for the installation of VOIP software into a vdi environment that call center users 500 miles away use. So that they won't have to have an actual phone on their desk anymore.

poo poo, it may work but it just sounds like a horrible plan to me.

Lync 2013 apparently supports it.

Doesn't make it a good idea, and it probably throws your VoIP COS/QOS rules out the window, but it's supported.

Lord Dudeguy fucked around with this message at 22:48 on Nov 20, 2013

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

go3 posted:

If it's important enough for them to call you, it's important enough that they'll leave a voicemail.

Also, any recommendations on home web filtering software for kids with parents that don't want to parent?

OpenDNS?

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Volmarias posted:

Until your children learn about VPNs and Tor, although it could be argued that at that point you have done your job.

The day the principal asks for a meeting because my daughter created an SSH tunnel w/ port forwarding at home to get around the school firewall will be the proudest goddamn day of my life. :fsmug:

Unless she did it with Windows/PuTTY and not *nix like a real woman. If that's the case, she's grounded for a week. :colbert:

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

MJP posted:

Hold music stories

Our Avaya IPOffice system only allows us to put an MP3 on an MP3 player and pipe it straight into the system via an audio cable. As a consequence, all hold music is just radio promo material done by our marketing dept.

But now, Lync 2010. Awwwww yes. IT's hold music has been changed to this delectable ditty.

My manager gave it "super-approval", as in "Approved, and if anyone gives you any guff about it I'll punch 'em in the throat."

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

evol262 posted:

I, too, often think "your tiny department that doesn't have the budget to buy new Windows licenses or keep up infrastructure should install Linux, because that will make things simpler".

Usually it goes hand-in-hand with "your tiny department that doesn't have the budget, but has high-budget requirements".

For example: my Domain Controllers must be fully up-to-date per PCI-DSS, but are in fact running Windows NT because gently caress licensing costs. So, with a gun to my head, I set up SAMBA, along with all the time needed for learning, firefighting, et al because my payroll is effectively a "sunk cost".

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

evol262 posted:

While I agree with this in principle, it's better for an experienced Windows shop to just run 2003 on top of Hyper-V or KVM than to bother setting up a Samba DC (I hope you're running Samba4) and fight for budget for licenses in 18 months than even try to get Linux in hand with no in-house expertise. I'm a Linux developer/sysadmin, and I get the rationale, but "install Linux, duh" is just not a solution to "we're a Windows shop with definite business requirements and no budget".

I hope you've documented the reasons you'll fail a PCI audit to your superiors, because auditors are not morons, and they'll discover your Windows NT DCs.

Oh I don't work there anymore, for obvious reasons.

We were PCI-D, so everything was self-audited "honor system". A lot of "Just check off 'Yes' to everything and let God sort it out".

They were trying to Linux-ize everything there for 50% of the users (LTSP on local library fire-sale Pentium 100s used as thinclients). The other 50% floated on OEM copies of Windows 2000/XP (on workstations bought in bulk off eBay) and that was "good enough".

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Migishu posted:


e: I was told to remove the link so I have removed the link. Would appreciate if others removed from their quotes

Why did this happen? It's free now.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

AlternateAccount posted:


Or just Remote Desktop back to your home machine or a colo box.

Or use DD-WRT/Tomato at home, and use a SSH Proxy Tunnel over port 443.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Collateral Damage posted:

Sounds like our archival storage for old virtual machines.. Copy them to a USB drive and put them in the non-fireproof safe inside the data center.

Ok, I'll bite. What's wrong with this practice?

I just started storing Decomm'd VMs on a large USB drive for a little bit of insurance. We were just recently bit by deleting a VM (poorly labeled server was also the KMS Server). Management went "LOL OH WELL" and we threw together a 2012 server fast enough, but I ordered a large USB drive anyway. No space left on the SAN for that kind of storage, no budget for NASes.

I'm making images of new physical boxes and putting them there as well (we don't backup servers that don't store irreplaceable data - most VMs are born from a template).

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Paladine_PSoT posted:

Keep your backups offsite

ConfusedUs posted:

Off-site backup

Oh, I must have missed that whole point. I take the drive off-site.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

blackswordca posted:

So a meeting request came in, from my manager

It's an exit interview. Treat it as such and don't burn bridges.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]
A passive-aggressive e-mail came in... to the entire department... CC'ing management:

quote:

Is there a problem with $software that you guys are aware of ?

That's right, don't explain what your problem is. Just assume that we're mind-readers and hope for the best, and CC management so that we look like your personal lapdog.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Malkar posted:

Tomorrow he's going to come in and dump holy water on it.

Nah I'm picturing something like this.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Paladine_PSoT posted:

I missed the original post so I'm assuming it was about Gwar.

In his honor I tried to switch the IT Helpdesk hold music to "Sick of You". :black101:

A ticket was rejected. :(

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

evobatman posted:

The answer is ALWAYS to show the same loyalty to the company that the company is showing to you, and never more than that.

Which is to say: "None whatsoever."

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

GoatShaver posted:

Who rebooted the DFS server on accident in the middle of the day while prepping for overnight work?

This loving guy right here.

Was it a clean restart or a dirty one?

If the latter, I hope you have your affairs in order.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

rolleyes posted:

Um... exactly how do you envisage this working without violating general relativity?

:edit: Ah gently caress, nevermind. That doesn't work either. :eng99:

Lord Dudeguy
Sep 17, 2006
[Insert good English here]
A speech impediment came in...

Exchange 2010 SP3 Unified Messaging was working great until HDAudio became a bad, bad thing.

Users who enunciate their "s"es are getting picked up by microphones and/or Lync phones with a slight whistle. When Exchange UM converts the audio as a greeting, the whistle comes out as a "sh".

"Thanksh for calling Bobsh Burgersh."

Record the greeting using garbage G711 over a cell phone or other phone system that doesn't do HDaudio/RTAudio, and it sounds fine. Don't enunciate your "s"es? You sound fine.

How in the gently caress am I going to fix this????

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Caged posted:

You've unlocked Sean Connery mode

Narrowing it down. It's not Lync. If I use a headset on a Lync phone/desktop that records at a lower frequency (i.e. "sounds like a telephone"), the VM greeting conversion sounds fine. It's absolutely UM desperately trying to convert a whistled "s" and failing.

I'm literally :psyduck:ing right now.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]
I'm pretty sure I solved my "Sean Connery" VoIP problem.

Lync "HDAudio" Phones and RTAudio records at 16-bit, 16kHz in "wideband" mode. This is what our users get all excited about when "It sounds like I'm right next to you! :j:"

Exchange UM requires a 16-bit PCM, 8kHz WAV to store the VM greetings, regardless of whether or not you set the Exchange UM codec to MP3 or WMA... that only applies to the upsampled/downsampled file you get in your e-mail as a Voicemail.

To test whether it was Exchange causing the problem, or just the nature of the sampling beast, I recorded my (and my wife's) voices using Audacity, trying to deliberately whistle our "s"es. Once I downsampled to the UM-required rate, two things happened:

1.) My "s"es turned to "sh"es.
2.) My wife's "s"es disappeared altogether.

I don't think there's anything else I can do at this point.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]
A ticket's been escalated...

Boss just shelled out the cash for Microsoft support because of Lync/Exchange Sean Connery mode.

I'm eager to find out what Microsoft has to say about this.

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

Sickening posted:

Cheap decision makers will use a workstation OS as servers. Its always more painful than its worth.

Running Windows 7 Professional 32-bit on a PowerEdge, here.

It's not because we're cheap. :smith:

Adbot
ADBOT LOVES YOU

Lord Dudeguy
Sep 17, 2006
[Insert good English here]

CommanderApaul posted:

I've also had a couple users want me to go downstairs where half the cubes are unoccupied and swipe toners from those printers so they don't have to get up and walk, literally, around the side wall of their cube to get their printjobs.

:smug: "Blah blah blah, CommanderApaul. Don't talk to me about cost savings. How much do you make? I'll bet we could get more savings there."

  • Locked thread