Powershell Megathread: Verb-Noun Scripting is the future

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Powershell Megathread: Verb-Noun Scripting is the future

«‹›94 »

Happiness Commando: Feb 1, 2002; $$ joy at gunpoint $$

What do you all use for version control? I've been trying to set up Git at work, and either Git is obtuse, I'm dumb, or both.

What I want is a repository that lives on an internal file share with the authoritative versions of those files actually stored there. I want something as close to seamless as possible for an ISE like VSCode (or whatever) to be pointed at that file share (repo, whatver) and natively edit those files with changes silently synced without having to muck around with the command line with every save. Our infrastructure is set up in such a way that scripts need to be ("need to be") locally edited on a variety of machines. Right now we use the ISE and frequently hear "wait a minute, this is the wrong file, I know I fixed this the other day, where is it" and queue the half hour of searching or re-coding.

While testing, I managed to get VSCode doing that with a cloud hosted repo on VSTS, but cloud hosted won't work in production for a variety of reasons. So now I'm trying a bunch of variations of

code:

git init
git clone
git remote add origin
git commit
git pull

and while I can get the remote repo synced locally, I can't figure out why the hell my local changes don't get pushed upstream. I must be missing one or two necessary steps but I dont know what the hell I am doing wrong. Alternatively, SVN seems like its designed more for centralized repositories, but the first time I mentioned it my boss said he didn't want to use it, so I dont know.

Help?

# ? Sep 29, 2018 19:53

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 06:57

mystes: May 31, 2006

Happiness Commando posted:

What do you all use for version control? I've been trying to set up Git at work, and either Git is obtuse, I'm dumb, or both.

What I want is a repository that lives on an internal file share with the authoritative versions of those files actually stored there. I want something as close to seamless as possible for an ISE like VSCode (or whatever) to be pointed at that file share (repo, whatver) and natively edit those files with changes silently synced without having to muck around with the command line with every save. Our infrastructure is set up in such a way that scripts need to be ("need to be") locally edited on a variety of machines. Right now we use the ISE and frequently hear "wait a minute, this is the wrong file, I know I fixed this the other day, where is it" and queue the half hour of searching or re-coding.

While testing, I managed to get VSCode doing that with a cloud hosted repo on VSTS, but cloud hosted won't work in production for a variety of reasons. So now I'm trying a bunch of variations of
code:
git init
git clone
git remote add origin
git commit
git pull
and while I can get the remote repo synced locally, I can't figure out why the hell my local changes don't get pushed upstream. I must be missing one or two necessary steps but I dont know what the hell I am doing wrong. Alternatively, SVN seems like its designed more for centralized repositories, but the first time I mentioned it my boss said he didn't want to use it, so I dont know.

Help?

You need to push the changes back to the server after committing them locally (git push / git push origin master / whatever). Also, git can be confusing in more complicated cases but it's really worth learning the basics like this. I wouldn't recommend using subversion in 2018.

# ? Sep 29, 2018 20:12

Happiness Commando: Feb 1, 2002; $$ joy at gunpoint $$

Jesus gently caress I don't understand any of this. Using VS Code to sync which does a pull and then a push.

code:


Git push origin master
Remote:error:refusing to update checked out branch: refs/head/master
By default, bla bla non bare repository will make the index and work tree inconsistent 
[remote rejected] master - >master (branch is currently checked out

I can use a bare repo as my remote (cloning the repo locally and then committing and pushing to the remote) works. loving finally. But the files all live on my workstation, since it's a bare repo.

OK. Make a bare repo, clone it to another folder on the server. Clone that other folder locally. Pull and push. Same loving error.

What the gently caress.

# ? Oct 1, 2018 21:54

mystes: May 31, 2006

Happiness Commando posted:

Jesus gently caress I don't understand any of this. Using VS Code to sync which does a pull and then a push.

I can use a bare repo as my remote (cloning the repo locally and then committing and pushing to the remote) works. loving finally. But the files all live on my workstation, since it's a bare repo.

OK. Make a bare repo, clone it to another folder on the server. Clone that other folder locally. Pull and push. Same loving error.

What the gently caress.

Why are you cloning it to another folder on the server and cloning from that?

Also, you would probably be better off asking git related questions in the General Programming Questions thread rather than this one.

# ? Oct 1, 2018 23:14

The Fool: Oct 16, 2003

There is actually a a git specific thread. It is fairly slow, but enough people have it bookmarked that questions get answered.

The Fool fucked around with this message at 23:31 on Oct 1, 2018

# ? Oct 1, 2018 23:28

Mr Crucial: Oct 28, 2005; What's new pussycat?

What's the best way to suppress error messages from non-Powershell object methods? I'm reading in an Excel file and them attempting to output each individual sheet as a CSV. It all works fine, it's just that some of my Excel workbooks don't have all of the sheets in them - at which point I get an ugly error message on screen.

code:

 $Excel = New-Object -ComObject "Excel.Application"
 $Workbook = $Excel.workbooks.open($file)

 Write-Host "Outputting temporary CSV files"

$Workbook.Worksheets.Item("Sheet 1").SaveAs("$OutputLocation\sheet1.csv",6) 
$Workbook.Worksheets.Item("Sheet 2").SaveAs("$OutputLocation\sheet2.csv",6) 
$Workbook.Worksheets.Item("Sheet 3").SaveAs("$OutputLocation\sheet3.csv",6)

I've tried the following, none of which have worked:

code:

$Workbook.Worksheets.Item("Sheet 3").SaveAs("$OutputLocation\sheet3.csv",6) | Out-Null
$Workbook.Worksheets.Item("Sheet 3").SaveAs("$OutputLocation\sheet3.csv",6) 2>&1 | Out-Null
[void]$Workbook.Worksheets.Item("Sheet 3").SaveAs("$OutputLocation\sheet3.csv",6)
$null = $Workbook.Worksheets.Item("Sheet 3").SaveAs("$OutputLocation\sheet3.csv",6)

# ? Oct 4, 2018 14:28

The Fool: Oct 16, 2003

Use try {} catch {} ?

# ? Oct 4, 2018 14:47

New Yorp New Yorp: Jul 18, 2003; Only in Kenya.; Pillbug

The Fool posted:

Use try {} catch {} ?

Or use -ErrorAction SilentlyContinue if you hate yourself and whoever has to maintain the script after you.

# ? Oct 4, 2018 16:52

Mr Crucial: Oct 28, 2005; What's new pussycat?

The Fool posted:

Use try {} catch {} ?

This worked, thanks.

New Yorp New Yorp posted:

Or use -ErrorAction SilentlyContinue if you hate yourself and whoever has to maintain the script after you.

This didn't work, because the method doesn't understand what -ErrorAction means - it's a ComObject method, not a native Powershell one.

# ? Oct 4, 2018 17:03

CzarChasm: Mar 14, 2009; I don't like it when you're watching me eat.

I'm trying to write a small powershell script that will go to active directory and return all records (users and distribution groups) and the boolean value of the attribute msExchRequireAuthToSendTo which will be either true, false, or not set.

Unfortunately something is broken. All I have so far is

code:

Get-ADUser -Filter {msExchRequireAuthToSendTo -eq $TRUE}

And that just returns me to the command prompt, returns no values.

Even if that did work, I would have to run it three times to get a version for all three values. And even then, I don't know what the argument for neither true nor false would be.

Is there some way to get just the account name or email address and the value of the attribute in a simple two column list?
Is powershell the best way to go about this or is there a better way?

I probably don't need to add that I'm new to powershell and don't have a lot of experience with it.

# ? Oct 4, 2018 17:54

Inspector_666: Oct 7, 2003; _{benny with the good hair}

I might be easier to just

code:

Get-ADuser -Filter * -Properties msExchRequireAuthToSendTo

and export the data to a CSV, especially if you're not sure the value of one of the things you want. Then you can filter and sort it in Excel.

# ? Oct 4, 2018 18:00

nielsm: Jun 1, 2009

Do you need to get this for every user and group object in AD? In that case just use Get-ADObject to return all user and group objects under the relevant OU. Or even just globally. Sort and filter the results afterwards.
Remember to use the -Properties parameter to specify which properties to fetch the values for.

E:f,b

# ? Oct 4, 2018 18:03

The Fool: Oct 16, 2003

CzarChasm posted:

I'm trying to write a small powershell script that will go to active directory and return all records (users and distribution groups) and the boolean value of the attribute msExchRequireAuthToSendTo which will be either true, false, or not set.

Unfortunately something is broken. All I have so far is
code:
Get-ADUser -Filter {msExchRequireAuthToSendTo -eq $TRUE}
And that just returns me to the command prompt, returns no values.

Even if that did work, I would have to run it three times to get a version for all three values. And even then, I don't know what the argument for neither true nor false would be.

Is there some way to get just the account name or email address and the value of the attribute in a simple two column list?
Is powershell the best way to go about this or is there a better way?

I probably don't need to add that I'm new to powershell and don't have a lot of experience with it.

Get-Aduser won't return groups, you should use get-adobject.

Following Inspector_666's advice, something like this should work:

code:

 Get-ADObject -Filter * -Property msExchRequireAuthToSendTo | Select DistinguishedName, ObjectClass, msExchRequireAuthToSendTo | Where {$_.ObjectClass -eq "group" -or $_.ObjectClass -eq "user"}

Then you can export that to a csv, or do whatever else you want.

# ? Oct 4, 2018 18:08

CzarChasm: Mar 14, 2009; I don't like it when you're watching me eat.

That works really well. Thank you very much for the help.

# ? Oct 4, 2018 19:44

FISHMANPET: Mar 3, 2007; Sweet 'N Sour
Can't
Melt
Steel Beams

Do people have thoughts on foreach vs ForEach-Object? Is one or the other more readable? Better performing? More powershelly?

I prefer foreach because it reads better to me but that could be the last vestiges of Python being my native language. My coworker uses ForEach-Object and it looks kinda dirty and messy but piping does seem more in line with doing things the Powershell way.

# ? Oct 9, 2018 23:44

The Fool: Oct 16, 2003

I prefer ForEach in scripts.

I have used ForEach-Object in one liners, but generally if my one liners is starting to get that complicated I probably need to unpack it a bit.

# ? Oct 9, 2018 23:47

anthonypants: May 6, 2007; by Nyc_Tattoo; Dinosaur Gum

FISHMANPET posted:

Do people have thoughts on foreach vs ForEach-Object? Is one or the other more readable? Better performing? More powershelly?

I prefer foreach because it reads better to me but that could be the last vestiges of Python being my native language. My coworker uses ForEach-Object and it looks kinda dirty and messy but piping does seem more in line with doing things the Powershell way.

I think this is a lot easier to grasp

code:

foreach ($x in $y) {
	Do-Thing $x
}

compared to any of these

code:

$y | ForEach-Object { Do-Thing $_ }

code:

$y | % { Do-Thing $_ }

code:

$y | foreach { Do-Thing $_ }

e: There's also this monstrosity in 4.0 or greater, apparently

code:

$y = @(1,2,3,4,5,6)
$y.ForEach({ Do-Thing $_ })

anthonypants fucked around with this message at 23:57 on Oct 9, 2018

# ? Oct 9, 2018 23:52

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

It depends on how far along in the code you need the context of the object. Like its been said, if its one line $_ isnt too bad but if its used in multiple places across forty lines of code its probably best it has a real name to avoid confusion.

Edit: wow that apostrophe conversion sure is a thing huh

Edit edit: I forgot to post earlier, has anyone had much experience with compounding variable values across recursion? For whatever reason passing a variable in, appending it, and then returning it lead to multiple repetitions of the same values in a way that struck me as odd. I instead ended up having it return an entirely new variable and appending that to the current variable.

PierreTheMime fucked around with this message at 00:55 on Oct 10, 2018

# ? Oct 10, 2018 00:51

sloshmonger: Mar 21, 2013

FISHMANPET posted:

Do people have thoughts on foreach vs ForEach-Object? Is one or the other more readable? Better performing? More powershelly?

I prefer foreach because it reads better to me but that could be the last vestiges of Python being my native language. My coworker uses ForEach-Object and it looks kinda dirty and messy but piping does seem more in line with doing things the Powershell way.

For some reason, there's a performance difference between those two seemingly identical functions.

https://blogs.technet.microsoft.com/heyscriptingguy/2014/07/08/getting-to-know-foreach-and-foreach-object/

code:

$time = (Measure-Command {
    1..1E4 | ForEach-Object {
        $_
    }
}).TotalMilliseconds
 [pscustomobject]@{
    Type = 'ForEach-Object'
    Time_ms = $Time
 }

$Time = (Measure-Command {
    ForEach ($i in (1..1E4)) {
        $i
    }
}).TotalMilliseconds
  [pscustomobject]@{
    Type = 'ForEach_Statement'
    Time_ms = $Time
 }

Running it now in PowerShell 5.1.17134 gives the foreach-object 95ms to count to 1000, whereas foreach does it in 17ms. Your system may be slightly faster and turn any difference negligible. I used to exclusively use foreach-object, but have converted to foreach since I evaluated that myself.

# ? Oct 10, 2018 04:47

anthonypants: May 6, 2007; by Nyc_Tattoo; Dinosaur Gum

sloshmonger posted:

For some reason, there's a performance difference between those two seemingly identical functions.

https://blogs.technet.microsoft.com/heyscriptingguy/2014/07/08/getting-to-know-foreach-and-foreach-object/
code:
$time = (Measure-Command {
    1..1E4 | ForEach-Object {
        $_
    }
}).TotalMilliseconds
 [pscustomobject]@{
    Type = 'ForEach-Object'
    Time_ms = $Time
 }

$Time = (Measure-Command {
    ForEach ($i in (1..1E4)) {
        $i
    }
}).TotalMilliseconds
  [pscustomobject]@{
    Type = 'ForEach_Statement'
    Time_ms = $Time
 }
Running it now in PowerShell 5.1.17134 gives the foreach-object 95ms to count to 1000, whereas foreach does it in 17ms. Your system may be slightly faster and turn any difference negligible. I used to exclusively use foreach-object, but have converted to foreach since I evaluated that myself.

And the method is supposed to be even faster.

# ? Oct 10, 2018 05:18

PBS: Sep 21, 2015

anthonypants posted:

And the method is supposed to be even faster.

PowerShell code:

function TestThing {   
    
    $time = @()
    $time2 = @()
    $time3 = @()

    $count = 0
    while ($count -lt 1000) {
        
        $time += (Measure-Command {
                1..1E4 | ForEach-Object {
                    $_
                }
            }).TotalMilliseconds
        
        $count++ 

    }

    [pscustomobject]@{
        Type    = 'ForEach-Object'
        Time_ms = ($time | Measure-Object -Average).Average
    }

    $count = 0
    while ($count -lt 1000) {

        $time2 += (Measure-Command {
                ForEach ($i in (1..1E4)) {
                    $i
                }
            }).TotalMilliseconds
        $count++
        
    }

    [pscustomobject]@{
        Type    = 'ForEach_Statement'
        Time_ms = ($time2 | Measure-Object -Average).Average
    }
     
    $count = 0
    while ($count -lt 1000) {
        $time3 += (Measure-Command {
                @(1..1E4).ForEach( {$_})
            }).TotalMilliseconds
    
        $count++
    
    }
    [pscustomobject]@{
        Type    = 'ForEach_Method'
        Time_ms = ($time3 | Measure-Object -Average).Average
    }

}

ForEach-Object: 69.9019716ms
ForEach_Statement: 7.292194ms
ForEach_Method: 39.3125347ms

# ? Oct 10, 2018 14:29

sloshmonger: Mar 21, 2013

PBS posted:

ForEach-Object: 69.9019716ms
ForEach_Statement: 7.292194ms
ForEach_Method: 39.3125347ms

anthonypants posted:

And the method is supposed to be even faster.

I had no idea the method even existed. Thanks for showing me that.

I've usually gone with functions or statements.

I guess you could say I'm not a... Method Man

# ? Oct 10, 2018 17:25

Volguus: Mar 3, 2009

sloshmonger posted:

I guess you could say I'm not a... Method Man

That would be a "methodic" or a "methodist".
I'd go with the religious one.

# ? Oct 10, 2018 18:55

anthonypants: May 6, 2007; by Nyc_Tattoo; Dinosaur Gum

Volguus posted:

That would be a "methodic" or a "methodist".
I'd go with the religious one.

Only registered members can see post attachments!

# ? Oct 10, 2018 19:10

FISHMANPET: Mar 3, 2007; Sweet 'N Sour
Can't
Melt
Steel Beams

fam just gently caress me up

code:

foreach ($i in (0..($array.Length-1))) {
    $array[$i]
}

# ? Oct 10, 2018 21:45

anthonypants: May 6, 2007; by Nyc_Tattoo; Dinosaur Gum

FISHMANPET posted:

fam just gently caress me up
code:
foreach ($i in (0..($array.Length-1))) {
    $array[$i]
}

# ? Oct 10, 2018 21:50

The Fool: Oct 16, 2003

FISHMANPET posted:

fam just gently caress me up
code:
foreach ($i in (0..($array.Length-1))) {
    $array[$i]
}

This is how I felt the first time I tried to do a for loop in Python.

# ? Oct 10, 2018 23:12

FISHMANPET: Mar 3, 2007; Sweet 'N Sour
Can't
Melt
Steel Beams

See I have a CS degree so I've done tons of programming in languages (C, C++, Python) with that for loop formulation, so the first few times I wrote one in Powershell I really had to scratch my head to wrap it around it.

# ? Oct 10, 2018 23:19

The Fool: Oct 16, 2003

I've always done loops in C/C++/C# as

code:

for (int i = 0; i <= 10; i ++) {}

Python threw me for a loopha because it was the first time I needed to do anything like this:

code:

for i in range(0, 10):

I don't have a CS degree, or do any real development though.

# ? Oct 10, 2018 23:29

Dirt Road Junglist: Oct 8, 2010; We will be cruel
And through our cruelty
They will know who we are

FISHMANPET posted:

fam just gently caress me up
code:
foreach ($i in (0..($array.Length-1))) {
    $array[$i]
}

Post/Avatar combo on point

# ? Oct 11, 2018 00:41

qsvui: Aug 23, 2003; some crazy thing

The Fool posted:

code:

for (int i = 0; i <= 10; i ++) {}

code:

for i in range(0, 10):

# ? Oct 11, 2018 03:14

Collateral Damage: Jun 13, 2009

Oh, we're talking about silly code?

https://twitter.com/flibitijibibo/status/1037011978600898560

# ? Oct 11, 2018 13:12

Judge Schnoopy: Nov 2, 2005; dont even TRY it, pal

Collateral Damage posted:

Oh, we're talking about silly code?

https://twitter.com/flibitijibibo/status/1037011978600898560

I'm really proud of myself for learning enough powershell to understand why this is funny.

# ? Oct 11, 2018 14:35

Methanar: Sep 26, 2013; by the sex ghost

I raise you

https://github.com/auchenberg/volkswagen

I loving lost it at INCEPTION_MODE

https://github.com/auchenberg/volkswagen/commit/ae5d3b6b825bbf4e83b968d0b8835746ffc0a465

Methanar fucked around with this message at 23:37 on Oct 11, 2018

# ? Oct 11, 2018 23:35

anthonypants: May 6, 2007; by Nyc_Tattoo; Dinosaur Gum

Today I used PowerShell to split up a giant CSV and apparently just using the > redirector generates UTF-16-encoded files, and they ended up ginormous

anthonypants fucked around with this message at 00:15 on Oct 12, 2018

# ? Oct 12, 2018 00:12

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

Apparently you can muck about with the local shell variables prior to redirecting to change the encoding format. Personally if you need to be certain I like the specificity of Out-File -Encoding $Format.

# ? Oct 12, 2018 00:59

anthonypants: May 6, 2007; by Nyc_Tattoo; Dinosaur Gum

PierreTheMime posted:

Apparently you can muck about with the local shell variables prior to redirecting to change the encoding format. Personally if you need to be certain I like the specificity of Out-File -Encoding $Format.

Yeah, I decided I'd write up a script to do it for people who aren't me, and decided that would be a good method. Easier than figuring out how to use StreamWriter or whatever, anyway.

# ? Oct 12, 2018 01:07

PierreTheMime: Dec 9, 2004; Hero of hormagaunts everywhere!; Buglord

anthonypants posted:

Yeah, I decided I'd write up a script to do it for people who aren't me, and decided that would be a good method. Easier than figuring out how to use StreamWriter or whatever, anyway.

Streams are actually pretty simple to work with and a lot faster. I posted about a similar topic a few pages back if you want an example I can provide one. I suppose it depends what �giant� means in your post, the ones I was working with were 20-40GB so the performance difference was significant.

# ? Oct 12, 2018 01:17

anthonypants: May 6, 2007; by Nyc_Tattoo; Dinosaur Gum

PierreTheMime posted:

Streams are actually pretty simple to work with and a lot faster. I posted about a similar topic a few pages back if you want an example I can provide one. I suppose it depends what �giant� means in your post, the ones I was working with were 20-40GB so the performance difference was significant.

This one is just over 1.4m rows, so it isn't actually too terrible. I'd actually like to use StreamWriter or whatever else in .NET to fix up our automation so we don't have to get an obscure error and manually run a script like this, but I don't want to change too many things while the senior guy is on paternity leave.

Anyway,

code:

# Use this script to blow apart a large file into smaller chunks.
# This script only works on flat files. If you've got an Excel spreadsheet, you'll have to use something else.
# Copy the big file somewhere and enter its path here:
#$bigfile = Get-ChildItem "C:\temp\file.csv"
# You could leave it on the network, but that's going to be really slow:
#$bigfile = Get-ChildItem "\\server\share\TempHold\file.pipe"
# Your desktop is probably a good place to put this file temporarily:
$bigfile = Get-ChildItem "$([Environment]::GetFolderPath("Desktop"))\file.csv"


$rowcount = 500000                  # each file will be split into this many rows
$header = 1                         # append this many header rows to each file
$outdir = $bigfile.DirectoryName    # output to the same directory as the file by default. probably a good idea to not set this to \\server\share
Write-Verbose "Reading $($bigfile.Name)..."
$content = Get-Content $bigfile     # this step will load the entire file into memory and will take a while
$numfiles = [math]::ceiling($content.Length/$rowcount)      # this many files will be generated


for ($i = 0; $i -lt $numfiles; ++$i) {
    $outname = "$($bigfile.BaseName)_$i$($bigfile.Extension)"   # "file.csv" => "file_0.csv", "file_01.csv", etc.
    Write-Verbose "Generating $outname... (file $($i+1) out of $numfiles)"
    if ($header -gt 0) {
        Out-File -FilePath "$outdir\$outname" -InputObject $content[($header-1)] -Encoding ascii
        Out-File -FilePath "$outdir\$outname" -InputObject $content[(($i*$rowcount)+$header)..(($i+1)*$rowcount)] -Encoding ascii -Append
    } else {
        Out-File -FilePath "$outdir\$outname" -InputObject $content[(($i*$rowcount))..(($i+1)*$rowcount)] -Encoding ascii -Append
    }
}

anthonypants fucked around with this message at 01:42 on Oct 12, 2018

# ? Oct 12, 2018 01:36

Adbot: ADBOT LOVES YOU

# ? May 15, 2024 06:57

Judge Schnoopy: Nov 2, 2005; dont even TRY it, pal

Can somebody here shed some light on the proper way to do this?

I have a collection of custom objects that have different properties. I'm using a single loop to say if target matches object.propertyA, but if propertyA doesn't exist, check if target matches object.propertyB. To accomplish this I'm doing a try / catch, but since I expect the error a lot, catch returns $null.

$collection = @[{name = A, a = a},{name = B, b = b}]
Foreach ($thing in $collection) {
Try{
If ($thing.a -match 'a'){return $true}
}
Catch {$null}
Try {
If ($thing.b -match 'b'){return $true}
}
Catch {$null}
}

For B, I expect the first if statement to throw an error complaining that property B.a doesn't exist. I want this error to be silent and for the script to move on.

I'm guessing I shouldn't be doing a try catch here but can't come up with the right answer to use instead.

Suggestions?

# ? Oct 12, 2018 01:38

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Powershell Megathread: Verb-Noun Scripting is the future

«‹›94 »