Powershell Megathread: Verb-Noun Scripting is the future

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Powershell Megathread: Verb-Noun Scripting is the future

kampy: Oct 11, 2008

Scaramouche posted:

Got some more goodies for you PS gurus. I'm trying to de-dupe that enormous file list mentioned previous (29,000 image files). I'm using this guy's script to find them:
http://blog.codeassassin.com/2007/10/13/find-duplicate-files-with-powershell/

Seems pretty good (goes by size and then md5), and the counts are really handy. Some of the files are duplicated over 1000 times. The problem I'm having though, is it's good to know the counts, but what I really need are all the duplicated file names themselves. Unfortunately the output of the script looks like this:
code:
Count   Name                  Groups
95	137 255 159 230 40 53 {127769_5.jpg, 127770_4.jpg...}
Where the ... is where the file list gets truncated. I >think< this could be controlled with Format-Table, but I'm not sure how I could get it to work, since in some cases (over 1000 file names) I'm probably going to run up against some kind of internal limit in powershell.

My other option is tweaking the script itself, since I don't actually need file counts and the 'name' column, all I would need is something like this:
code:
ID   FileName
1    127769_5.jpg
1    127770_4.jpg
1    127771_3.jpg
2    131110_1.jpg
2    131111_2.jpg
2    131112_3.jpg
etc.
Where the 'ID' just denotes that 'yes all of these files are the same'. The second option is actually more useful to me, because then I could relatively quickly turn this list into SQL, instead of having to harvest out several comma-delimited lists like the Format-Report solution does. My tweaking has not found success thus far though, resulting only in errors, returning no files, or returning that all files are identical. My questions are thus:
1. Do you guys think that I should be concentrating on Format-Report or just tweaking the script to get my desired output? Can Format-Report columns even go that large?
2. Anyone have any advice on how to get the desired output in the second example from the script linked above?

Thanks in advance for any help.

I would just modify the script a bit, or perhaps just run something like:

code:

$dupes = .\unmodifiedscriptfromtheweb.ps1 c:\whatever\path
foreach ($dupe in $dupes)
{
	foreach ($d in $dupe.Group)
	{
		# Change $d.Name to $d.FullName for the full path.
		"{0} *{1}" -f $dupe.Name, $d.Name
	}
	write-host ""
}

You could also modify the script a bit so that it produces a prettier md5 output by modifying the Get-MD5 function to something like this:

code:

function Get-MD5([System.IO.FileInfo] $file = $(throw �Usage: Get-MD5 [System.IO.FileInfo]�))
{
# This Get-MD5 function sourced from:
# [url]http://blogs.msdn.com/powershell/archive/2006/04/25/583225.aspx[/url]
$stream = $null;
$cryptoServiceProvider = [System.Security.Cryptography.MD5CryptoServiceProvider];
$hashAlgorithm = new-object $cryptoServiceProvider
$stream = $file.OpenRead();
$hashByteArray = $hashAlgorithm.ComputeHash($stream);
$stream.Close();

## We have to be sure that we close the file stream if any exceptions are thrown.
trap
{
if ($stream -ne $null) { $stream.Close(); }
break;
}

foreach ($byte in $hashByteArray)
{
	$returnme += $byte.ToString("x2")
}
return $returnme
}

Edit:
You could also do something like this:

code:

$output = ""
$dupes = .\unmodifiedscriptfromtheweb.ps1 c:\whatever\path
foreach ($dupe in $dupes)
{
	foreach ($d in $dupe.Group)
	{
		$whatwewant = @{md5=$dupe.Name; filename=$d.Name}
		[array] $output += new-object psobject -property $whatwewant
	}
}
$output

kampy fucked around with this message at 10:32 on Nov 10, 2011

# ¿ Nov 10, 2011 10:17

Adbot: ADBOT LOVES YOU

# ¿ Apr 30, 2024 16:12

kampy: Oct 11, 2008

Nebulis01 posted:

Stupid question time again, regarding the following script

It throws receive the following errors

code:

Remove-Item : The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
At line:1 char:125
+ Get-ChildItem -Path C:\backups\servername\flatfile -Recurse |  where { ((get-date)-$_.creationTime).days -ge 4}  | Remove-Item <<<<  -recurse -force
    + CategoryInfo          : WriteError: (C:\backups\servername\flatfile\2012-04-01:String) [Remove-Item], PathTooLongException
    + FullyQualifiedErrorId : RemoveItemIOError,Microsoft.PowerShell.Commands.RemoveItemCommand

Remove-Item : Cannot remove item C:\backups\servername\flatfile\2012-04-01\c\Documents and Settings\user_ad\Local Settings\Temp\hsperfdata_user_ad: Access to the path is denied.
At line:1 char:125
+ Get-ChildItem -Path C:\backups\servername\flatfile -Recurse |  where { ((get-date)-$_.creationTime).days -ge 4}  | Remove-Item <<<<  -recurse -force
    + CategoryInfo          : InvalidArgument: (hsperfdata_user_ad:DirectoryInfo) [Remove-Item], ArgumentException
    + FullyQualifiedErrorId : RemoveFileSystemItemArgumentError,Microsoft.PowerShell.Commands.RemoveItemCommand
Remove-Item : Cannot remove item C:\backups\servername\flatfile\2012-04-01\c\Documents and Settings\user_ad\Local Settings\Temp: The directory is not empty.

At line:1 char:125
+ Get-ChildItem -Path C:\backups\servername\flatfile -Recurse |  where { ((get-date)-$_.creationTime).days -ge 4}  | Remove-Item <<<<  -recurse -force
    + CategoryInfo          : WriteError: (Temp:DirectoryInfo) [Remove-Item], IOException
    + FullyQualifiedErrorId : RemoveFileSystemItemIOError,Microsoft.PowerShell.Commands.RemoveItemCommand
Remove-Item : Cannot remove item C:\backups\servername\flatfile\2012-04-01\c\Documents and Settings\user_ad\Local Settings: The directory is not empty.

Bit of a late reply, but the first error you're running into is the good ole NTFS(?) limitation on path length, which can not exceed ~250 characters. It may also be causing takeown not to work properly which would then account for the two other errors.

You could for example, create temporary file shares in a subdirectory of the path (one at the 2012-04-01 directory in your example case might be good) and do the file removal in those which bypasses the issue.

Quick example of that:

code:

$share = "c:\temp"
# WMI might be a better idea for this but:
invoke-expression ('cmd.exe /c net share hidden$="{0}" /grant:"everyone,full"' -f $share)
pushd \\127.0.0.1\hidden$
rmdir "a path"
popd
invoke-expression ('cmd.exe /c net share hidden$ /delete /y')

Nebulis01 posted:

I tried to end run around Remove-Item by creating the variable $olddata and passing that to the a command prompt invoking RD(rmdir), but that also errors out
code:
$olddata = (Get-ChildItem -Path $testFolderPath -Recurse | where { ((get-date)-$_.creationTime).days -ge 4} | Where {$_.psIsContainer -eq $true} )
    #Remove-Item -Path $olddata -recurse -force 
    Invoke-Expression "rd $olddata /q /s" 
My brain is fried, any idea how I can get Powershell to delete these leftovers?

I believe Invoke-Expression will try to run the powershell version of the command, if it exists, so if you want to run the command prompt version of rmdir, you'd want to do something like this:

code:

# probably a good idea to substitute in the variable names too, this replaces {0} with contents of $olddata:
Invoke-Expression ('cmd.exe /c rd "{0}"' -f $olddata)

Also using foreach to loop through the $olddata variable might be a good idea in this case, since it may return more than one directory.

# ¿ Apr 11, 2012 07:51

kampy: Oct 11, 2008

Nebulis01 posted:

Any ideas on why the following returns items that are from yesterday instead of just the stuff 5 days old and older?
code:
$targetFolder = "c:\backups"
Get-ChildItem -Path $targetFolder -recurse | WHERE {($_.CreationTime -le $(Get-Date).AddDays(-5))} | Remove-Item -recurse -force -whatif 

Have you tried with LastWriteTime instead of CreationTime? It may be that the CreationTime does not reflect the correct date for whatever reason, I'd recommend checking the attributes for the incorrectly matched files with

code:

select fullname, creationtime, lastwritetime

in place of the Remove-Item part.

# ¿ Jun 23, 2012 16:23

kampy: Oct 11, 2008

Jelmylicious posted:

With parameters, you couldt use a validateset:

code:

param
    (
    [Parameter(Mandatory=$True)]
    [validateset('Mars','Mordor','Sodom','Gomorra')]
    [string]$City
    )

And for more help and samples with those, check

code:

help about_functions_advanced_parameters

# ¿ Jun 27, 2012 06:24

kampy: Oct 11, 2008

Siets posted:

1. User supplies name of a group.
2. Name is used to generate an AD group based on the name.
3. A series of directories are created on the share based on the name.
4. NTFS permissions of the directories are modified based on the AD group that was made in step 2.

What I'm thinking might work for this is to wrap up my $dirAcl.AddAccessRule($rule) methods inside of a Try-Catch that itself is nested inside a Do-While loop based around ending at non-detection of an error. Then I can set $ErrorActionPreference to "SilentlyContinue" and be on my merry way. Problem is, I'm pretty new to error handling (bad, I know) and would just like some outside ideas on a cleanly way to do this. Is there a better way than my Try-Catch / Do-While idea? I'm pretty open.

Are you using New-ADGroup from the ActiveDirectory module to create your group initially? If you are, you could do something like:

code:

$group = New-ADGroup -PassThru @and_the_rest_of_your_variables_as_before

#and sample rule to illustrate:
$write = [System.Security.AccessControl.FileSystemRights] "Write"
$inheritance = [System.Security.AccessControl.InheritanceFlags] "ContainerInherit, ObjectInherit"
$propagation =  [system.security.accesscontrol.PropagationFlags] "None"
$access = [System.Security.AccessControl.AccessControlType] "Allow"

$rule = New-Object System.Security.AccessControl.FileSystemAccessRule($group.sid, $write, $inheritance, $propagation, $access)

$group.SID will contain the SID of the newly made group, which you can then use when you create your $ruleRead and $ruleMod variables. Then you should be able to apply the rules instantly without needing to wait for the group to propagate everywhere.

Also when doing try-catch, don't set $ErrorActionPreference, use -ErrorAction with the command instead, that way you will not accidentally let errors fail silently in places where you don't want them to.

# ¿ Jul 5, 2012 16:33

kampy: Oct 11, 2008

angrytech posted:

Alright, I'm using a DirectorySearcher to return a list of computer accounts from AD. Is there a way to figure out the last date that these accounts were modified on?

They have a whenChanged property you can use. Sample:

code:

$root = [ADSI] ''
$searcher = New-Object System.DirectoryServices.DirectorySearcher($root)
$searcher.filter = "(&(sAMAccountName=*`$))"
$test = $searcher.FindAll()
foreach ($comp in $test) { $comp.properties.whenchanged }

# ¿ Jul 21, 2012 10:16

kampy: Oct 11, 2008

String formatting is what you'll want to use in single quote strings:

code:

$var = 5
'It is: "{0}"' -f $var

# ¿ Jul 31, 2012 08:22

kampy: Oct 11, 2008

Powerful Two-Hander posted:

I was trying to use System.net.httpwebrequest to do this but get 401 errors returned and I can't figure out (because I literally know nothing about what I'm doing and just copy poo poo I find on technet - this may well be the problem here) how to get the existing session credentials sent through to pass authentication. The user account it's running from has access to the site and I thought that by default -usecurrentcredentials is set to true so it shold work. I'd really like to avoid having to specify a username and password in the script as it's just one more thing to have to change if the password ever gets modified.

Is there something obvious/easy I'm missing here?

Yeah, kind of. Usedefaultcredentials is false by default, so you'll need to set it to true.

code:

$req = [System.Net.HttpWebRequest]::Create("https://companyweb.whatever")
$req.UseDefaultCredentials = $true
$res = $req.GetResponse()

# ¿ Aug 6, 2012 12:10

kampy: Oct 11, 2008

Since you can pipe objects to Add-ADPrincipalGroupMembership, you could also do something like this:

code:

Get-ADUser -Filter * -SearchBase "OU=Test,DC=domain,DC=local" | Add-ADPrincipalGroupMembership -MemberOf "CN=group,DC=domain,DC=local"

e: using -Filter {objectClass -eq "user"} is also redundant with Get-ADUser

kampy fucked around with this message at 19:32 on Aug 8, 2012

# ¿ Aug 8, 2012 18:04

kampy: Oct 11, 2008

stubblyhead posted:

Functionally there's no difference between doing
code:
$users = Get-ADUser -Filter * -Searchbase "foo"
Foreach ($user in $users) {
  Add-ADPrincipalGroupMembership -MemberOf "bar" -Identity $user
}
and what kampy did; kampy's way is just more concise and easier to read once you get your legs under you. Let's pretend for a second that we wanted to add a single user into a bunch of groups, though. The -MemberOf parm doesn't take pipeline input, so you would need to do a foreach or similar to add the user to all your groups.

As it turns out, there is a small difference between my example and using a foreach as above.

Both ways work identically when adding users from an OU to a group as long as none of the users is a member of said group. However if one of the users fetched by Get-ADUser is already a member of the group, none of the users will get added when passing the user objects through a pipeline.

Assuming I read the Trace-Command logs correctly, it looks like Add-ADPrincipalGroupMembership passes all the users on to Add-ADGroupMember in a single call which then raises an exception if one of the users has previously been added, causing none of the users to be added. Since the foreach loop ends up calling Add-ADGroupMember individually, exceptions are raised for the users that are members already and the rest are correctly processed.

So pipelines can be effective, but they don't always end up doing what you expect. Testing with multiple scenarios is important

# ¿ Aug 9, 2012 12:17

kampy: Oct 11, 2008

stubblyhead posted:

Are you sure that is correct? The way I understand it, only one object gets passed from the pipeline at a time, so the cmdlet wouldn't have any notion that there were other objects, much less that some were already in the security group. It doesn't look like -Identity even accepts an array as input.

Yep, that's the way it works with PowerShell 2.0 on Server 2008 R2 in any case. It's not the way I would expect piping objects to work either, but I assume it's by design because according to help Add-ADPrincipalGroupMembership it will use a single AD operation to do it's business.

# ¿ Aug 9, 2012 17:55

kampy: Oct 11, 2008

stubblyhead posted:

Am I going about things in entirely the wrong way, or is handling HTTP requests a very cumbersome process in Powershell? For instance, if I wanted to get google's homepage, I could do something like this:

Is there really no simpler way to do this?

That's pretty much the way it goes in PowerShell v2, in v3 you can use Invoke-WebRequest http://www.google.com

# ¿ Sep 19, 2012 10:02

kampy: Oct 11, 2008

MJP posted:

Edit: bada bing. get-distributiongroupmember -identity "groupname" | format-table Name,PrimarySMTPaddress > nameslist.csv did the trick and some text to columns made it legible.

I don't have the exchange cmdlets installed, but you might want to consider using select and Export-Csv there instead of format-table, something like this should work:

code:

get-distributiongroupmember -identity "groupname" | select Name,PrimarySMTPAddress | Export-Csv -Path nameslist.csv -NoTypeInformation

# ¿ Sep 28, 2012 20:17

kampy: Oct 11, 2008

Defghanistan posted:

Does anyone know how to find the file associated with the function currently loaded into memory?

For example let's say I load a function:

***begin***
cd D:\scripts
.\renameserver.ps1

Rename-Computer $name $creds
***end***

Is there any way to find the source file that Rename-Computer was loaded from? I've been digging for this and haven't found it anywhere but it seems really useful.

code:

(Get-Item Function:\Rename-Computer).ScriptBlock.File

# ¿ Nov 15, 2012 17:49

kampy: Oct 11, 2008

MJP posted:

So am I looking at this as a possible script:

code:

\\servername\c$\tools\delprof2 /c:\servername /p

$x = [array]$excludeUsers = "Citrix","admin","Track"
$profiles = get-Childitem C:\users -directory  -exclude $excludeUsers
$y = Delete inactive profiles on 'Servername'? (Yes/No) #Say Y to start the process
  if ($x) {Write-Host "N"} #Say N since this account should never be deleted
  elseif {Write-Host "Y"} #say Y to delete all other accounts not specified in $x

Nope, that won't really help you.

With delprof2, wouldn't you be better off by passing /ed:citrix /ed:admin /ed:track arguments to it directly to exclude certain profiles instead of attempting to use powershell to do so?

# ¿ Nov 17, 2012 12:22

kampy: Oct 11, 2008

AreWeDrunkYet posted:

What am I doing wrong?

Your script works fine, so I assume the rules you're trying to remove are inherited? If that's the case, they can not be directly removed from a subdirectory. You'd have to remove them from the source or remove inheritance from the subdirectory and copy over the rules you want to keep.

# ¿ Nov 20, 2012 17:44

kampy: Oct 11, 2008

MJP posted:

Sorry to continue harping on this, but I did some research and couldn't quite find a native PS cmdlet to do this - there's a Scripting Guy entry about making an .hta to find it but I'm supposed to do this in one file alone.

Anyone?

Judging from the error message you do not have the custom function Create-UiList that adaz posted copypasted in your DisplayUsers.ps1 or if you do, it is below the code that tries to execute it and therefore it can not be found. Either way you'll want to go and fix your script so that the function is correctly created above the $users line, go get it from:

adaz posted:

code:

#################
# Create-UIlist #
#################
# ..snip..

# ¿ Nov 28, 2012 22:11

kampy: Oct 11, 2008

stubblyhead posted:

but it doesn't work if $myCollection only has a single element. I'm not sure if this is how the PowerCLI cmdlets are written or if it's just how Powershell works, but the Get-Cluster cmdlet will return a single object if there's only one that matches the criteria, and a collection of them if there's multiple. The Cluster object doesn't have a Length property, so Get-Random thrown an error because the parameter gets passed a null value. I'd prefer not to do a check on the length before doing this if possible. Is there any way I can make it handle a single object as a collection with length 1, or is there just a better way to approach this altogether?

You could also try forcing $myCollection to be an array, that way it will always have a length.

code:

[array] $myCollection = Whatever-Command

# ¿ Feb 6, 2013 08:40

Adbot: ADBOT LOVES YOU

# ¿ Apr 30, 2024 16:12

kampy: Oct 11, 2008

Swink posted:

Is there a prettier method of getting the user to make a selection? At the moment I use this:

Parameters for your script/function would be the best way really, here are a couple of simple examples, check help about_functions_advanced_parameters in powershell for more things you could do:

code:

function Show-Thingy
{
    param
    (
        [Parameter(Mandatory=$true)]
        [ValidateSet("Melbourne","Sydney","Brisbane")]
        [String] $Location
    )

    $Location
}

function Show-OtherThingy
{
    [CmdletBinding(DefaultParameterSetName="Melbourne")]
    param
    (
        [Parameter(ParameterSetName="Melbourne")]
        [Switch] $Melbourne,
        [Parameter(ParameterSetName="Sydney")]
        [Switch] $Sydney,
        [Parameter(ParameterSetName="Brisbane")]
        [Switch] $Brisbane,
        [String] $Name="nobody"
    )
    Process
    {
        "{0}, said {1}" -f ($PSCmdlet.ParameterSetName, $Name)

        if ($PSCmdlet.ParameterSetName -eq "Sydney")
        {
            "Hello $Name!"
        }
    }
}

# ¿ Apr 17, 2013 19:44

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Powershell Megathread: Verb-Noun Scripting is the future