Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


ChubbyThePhat posted:

Wew 1903 is finally out. Made a service ring to test on my and a coworker's machines. Obviously not looking to get this in production but hey new stuff!

Same, internal test ring wants it already and it's passed my basic "will it install and not gently caress everything up" pass so I'll let them figure out what's wrong with it.

Adbot
ADBOT LOVES YOU

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


ChubbyThePhat posted:

sometimes Cortana decides to gently caress right off

I wish Cortana would gently caress off forever

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Mr. Clark2 posted:

Windows licensing question(s):

I have 200'ish PCs that were all originally purchased with OEM Win10 Pro licenses and I would like to move them all to Win10 Enterprise in order to take advantage of some of the features like Applocker. Is this path possible? What is the easiest way to acquire the Enterprise licenses? Who do I even call to get a quote? Why has MS made licensing so difficult?
Our Office 365 licenses (E1 and E3) are all non-profit pricing, so hopefully MS offers something similar on the Windows side. Gracias.

You want Microsoft 365 E3: https://www.microsoft.com/en-ca/microsoft-365/enterprise/e3?activetab=pivot%3aoverviewtab

Your Office 365 would get rolled into the Microsoft 365 package and you'd get all the other stuff you want.

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


The Semi-Annual Channel for Windows Server is also being discontinued with Server 2022 so it’s probably not worth it to look into using it

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


AlternateAccount posted:

Aha, thank you. So really, it's Server 2019 that's the primary "full featured' release, and it's getting a proper replacment with 2022? What a weird detour this other business was, then.

That's right. 2016, 2019, and the upcoming 2022 are the LTSC versions of Windows Server. The other ones were part of an attempt to use Windows Server for some sort of container stuff and I don't think it caught on enough for them to continue. Windows Server container images are still going to be a thing, but they will be based off 2022 going forward. I think that they are trying to move the Windows Server container stuff to Azure now, where they can manage the updates and such.

Those other versions had all sorts of rules like they could only run server core, or something they called Nano Server. Like the other poster said, they had very narrow use cases.

e: for reference in terms of Widows Server to Windows 10 feature set mapping:

Server 2016 = Windows 10 1607 LTSC
Server 2019 = Windows 10 1809 LTSC
Server 2022 = Windows 10 21H2 LTSC (supposedly)

This is mostly useful to know what the UI looks like and such. Avoid 2016 like the plague if you can since it has the Windows 10 1607 updates servicing which is extremely slow.

Number19 fucked around with this message at 20:12 on Aug 13, 2021

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


I have a workstation that absolutely will not apply group policy. C:\Windows\GroupPolicy or whatever is empty and gpupdate fails saying it can’t read the policy from the domain controller despite my being able to browse to the policy in the policy store on all DCs.

If I disable the policy it complains about in gpupate another one fails, if I disable that one yet another fails, and so on

I’ve decided the computer is haunted and I’m going to wipe it since I’ve wasted more than enough time on it and it’s not that important but I figured I’d pop it in here to see if anyone has an idea about how to get this to work? The only thing I can think of is to remove it from the domain and add it again but given the empty directory I don’t think that’ll do it

Any ideas?

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Yeah that’s where I’m at. I tried for an hour or so and I’ve hit my “it’s faster to rebuild it” threshold

It’s legacy anyways so once I get the owners to evacuate any worthwhile data it’s going into the bin

E: I might keep it around as a side project to plink around with when I have free time blocks since this kind of thing is exactly what I need to hone and refine troubleshooting skills but there’s no way it’s ever going back into production

Number19 fucked around with this message at 05:59 on Jul 8, 2022

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Number19 posted:

I have a workstation that absolutely will not apply group policy. C:\Windows\GroupPolicy or whatever is empty and gpupdate fails saying it can’t read the policy from the domain controller despite my being able to browse to the policy in the policy store on all DCs.

If I disable the policy it complains about in gpupate another one fails, if I disable that one yet another fails, and so on

I’ve decided the computer is haunted and I’m going to wipe it since I’ve wasted more than enough time on it and it’s not that important but I figured I’d pop it in here to see if anyone has an idea about how to get this to work? The only thing I can think of is to remove it from the domain and add it again but given the empty directory I don’t think that’ll do it

Any ideas?

I'm bumping this because this has now happened to a few other computers and I'm out of troubleshooting steps.

Some new stuff I have discovered:

  • there are no common threads between the computers that I can find. They don't have a common piece of software, they are in different OUs with different policies linked
  • the problem seems to be specifically that the computer can no longer access SYSVOL. I used PSExec to open a command prompt as system and then did a pushd to \\domain.name\sysvol and it can't access it. On a non busted computer this works just fine
  • for the above, the fault appears to be only using FQDN. If I use the NETBIOS name I can browse SYSVOL
  • removing from the domain and rejoining does not seem to help, even rejoining to a new computer account
  • using DISM to restore health and using SFC do not help
The new ones seemed to fail completely at random. One week they refresh, the next they don't. I've had 1 go bad, then 3 at once, then 2. In the end it's still faster to reimage the computers than debug this but I'm hoping this rings a bell. It really just seems like the computer itself caches something in some weird way and then can never access SYSVOL again

It's a truly baffling issue and I'd love to know why the gently caress this happens. Google searches have yielded nothing that has helped. I almost want to pay MS for a support ticket but I don't think anyone there will know either. Does anyone have any ideas for poo poo I haven't tried yet?

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


The Fool posted:

it's dns op

I would have thought that as well but the system account can resolve all the domain and DC FQDNs just fine.

e: very specifically when trying to connect to SYSVOL as system, the error returned is: The specified network password is not correct.

It's definitely an authentication issue, but I have no idea why

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Thanks Ants posted:

Does the network path between working PCs and broken PCs differ?

Some are VPN clients, some are LAN. That was my first guess as well but I had some of each break around the same time.

e: to be clear: 2 of the borken ones were in the same VLAN, although they do not share a switch. Each switch's path to the core is identical though

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Submarine Sandpaper posted:

Kerberos works with netbios. I don't know enough to know why when you use the fqdn it is defaulting to ntlm, maybe.

The error from the Group Policy client uses the FQDN for policy refreshes:

quote:

The processing of Group Policy failed. Windows attempted to read the file \\fqdn\SysVol\fqdn\Policies\{39204A9B-B3A0-4C46-AFF1-E7437F41A346}\gpt.ini from a domain controller and was not successful. Group Policy settings may not be applied until this event is resolved. This issue may be transient and could be caused by one or more of the following:

a) Name Resolution/Network Connectivity to the current domain controller.

b) File Replication Service Latency (a file created on another domain controller has not replicated to the current domain controller).

c) The Distributed File System (DFS) client has been disabled.

where fqdn is the proper value

skipdogg posted:

What's your imaging process like? Are you sysprep'ing your machines?

This would have been done on a fresh out of the box non-imaged machine, like all the others which are not showing this problem. The very first one was imaged using SCCM and was definitely sysprepped

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Submarine Sandpaper posted:

Kill your tickets on a broke and see if you get any on a new attempt.

I think it's dns tho

I have purged

skipdogg posted:

Have you verified all your DNS records for the domain controllers? Make sure there's no old DC information hanging out, and sites and services are setup and up to date.

Also setup a packet capture on a busted machines and see what it's doing and where it's trying to get its info from.

the computer account tickets and that did not help

All my records are up to date and I have made sure there are no records for dead DCs anywhere

e: i'll setup Wireshark on one of them in a few and see if I can get a capture of wtf is happening

Number19 fucked around with this message at 23:22 on Nov 28, 2022

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


The Wireshark at least showed a bit more useful info. I am getting a KRB5KDC_ERR_PREAUTH_FAILED for the computer account policy refresh. I think this eliminates DNS as the fault and seems to confirm what I saw before with the pushd command:

quote:

C:\WINDOWS\system32>pushd \fq.dn\sysvol
The specified network password is not correct.

Now I need to figure out why the computer account is having Kerberos failures. It definitely seems like the computer password is wrong now somehow, but I've tried a computer account password rotation with no luck. What a bizarre problem.

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Thanks Ants posted:

Does klist show a difference between a working and a non-working machine?

Edit: Wait, computer account. Do you have some script being pushed running as the computer user that's attempting to use network resources and getting the machine locked out after failing?

No, no scripts to my knowledge. I wouldn't do something like that anyways since it's a Bad Idea.

I don't think it's an account lockout issue anyways since \\NETBIOS\sysvol works while \\FQDN\sysvol does not.

I think these computers are haunted

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


This has taken an interesting turn this morning. I think I've isolated the "what" here but have no idea as to the "why" or how to fix it.

I did a couple more tests and this is not isolated to SYSVOL. It's any DFS path accessed using the FQDN. Using the same methodology to get a command prompt as system, I ran "start \\fqdn\sysvol" and after some time was greeted by an "Enter network credentials" prompt which then has the username of the former workstation user. As in, the computer account is trying to login in using DOMAIN\user.name instead of DOMAIN\COMPUTER_NAME$. On a working workstation, this goes straight through.

It really looks like that particular DFS logon path has a bad cached credential. Running the same test for \\DOMAIN\sysvol goes straight through.

The question is "how the gently caress did this happen?" and "how do I make it stop doing this?"

What a weird fault.

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Well goons, I have this sorted out. I still don't exactly know the "why" but I have the "what" and the "how to fix" so I'm good for now.

So far as I can tell, this is what's been happening:

  • user changes their password
  • the Kerberos delegation the computer account uses to access DFS shares doesn't get updated with the new credentials (this is the "why" I haven't sorted out yet)
  • the computer continues to use the old user password for delegation which no longer works
  • DFS connections for the computer account to the FQDN stop working
  • Windows seems to just give the gently caress up and just have access to DFS via FQDN be broken forever
  • Group Policy fails to update, which is the only thing the computer account is using this delegation for

The fix is to login as the local admin (using the LAPS password), use PSexec to elevate to system, connect to the DFS share (using start \\fdqn\sysvol so it pops up the login prompt), have the user enter their new password (sometimes required twice). This fixes the fault and GPOs update again.

I mean...what the gently caress but at least I have a fix.

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


GreenNight posted:

Were the DC's patched recently? Been reading about authentication issues with the latest Windows patch.

Yes. but I had pre-verified that we would not be affected by the KRB issues. This is definitely some sort of weird race condition or some other stupid poo poo whereby Windows had it set to "only use THESE credentials for delegation" and ones those had changed there was no visible way to update them.

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


If it were that, it would have blown up two weeks ago and also would have been more wide spread.

I had this issue happen months ago, well before the patch. I’m pretty sure that’s not the root cause here

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Thanks Ants posted:

Are your DFS namespace servers all healthy? Are the DNS zones for the namespace as they should be?

The namespace servers are healthy and DNS works right. This really seems like a case where Windows somehow got configured (or configured itself) for the computer account to access DFS via the FQDN using one explicit set of credentials and then had the computer account get stuck when those credentials were no longer valid. The very first fault I had with this was because the primary user (who's account would have been who's credentials the computer account got stuck like this) was disabled because they left the company.

I can definitely fix it now at least but it really should not get stuck like this. I get why it's stuck since the computer account has no way to ask the user to re-authenticate themselves but it really should throw those credentials out and start over if they are obviously broken.

We did a password reset wave not too long ago so I expect I will see many more failures as more cached credentials get "stuck" like this.

e: also of note, this problem persisted through a domain leave and rejoin to a new computer account object, which seems extra absurd to me

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


To put the capstone on this: the cause here was domain credentials for the FQDN being saved in the computer account's credential manager. Once the password was no longer valid, the computer account was trying to use these cached credentials to access the DFS namespace and failing. By using psexec and deleting that credential, the computer is unstuck and back to working properly.

How did those get set for only a few computers? Truly a mystery and not one I plan on trying to figure out right now. It really seems like Windows should not allow this to happen and should definitely fall back to not using those credentials when they don't work.

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Thanks Ants posted:

It sounds like someone's tried to script something in the past and targeted the wrong context

That's about all I can guess. The list of affected users has no common thread to them, so I can't figure out what this diverse group would all have done. I think that part of the RCA is going to remain a mystery. If it pops up again I will be able to dig in by tracking recent changes.

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


snackcakes posted:

For license assignments I recommend leveraging dynamic AAD groups as much as you can. It's made my life a heck of a lot easier but obviously everyone's environment/requirements are different

I just did something like this and it is 100% the way to go. It might take a bit to figure out your rule syntax (the basic rule editor is nearly useless) but one you have a framework in place you suddenly stop having to worry about a lot of things that you did manually before.

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


Glad I caught up on this thread. We have a couple of new laptops which have Optimizer and people were wondering why some Youtube videos sounded strange. Turns out it's Optimizer's "Remove others' background noise" setting. Thanks Dell!

Adbot
ADBOT LOVES YOU

Number19
May 14, 2003

HOCKEY OWNS
FUCK YEAH


kiwid posted:

Are Yubikey's reusable, as in if the employee leaves I can somehow reassign it to another user?

They are, yes. But they are also not all that expensive and we treat them as a consumable. If someone get a Yubikey from us, it's theirs forever. You can't stop them from using it for non-work accounts once they have it so it just becomes a personal item for them and we don't want it back.

They are also great and I wish I could get more people to take them. I use them for all my daily and admin accounts and it's so much easier to use.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply