Objects in Flight: Rewiring Your Mental Model of the PowerShell Pipeline
Photo: NASA, Public domain, via Wikimedia Commons
Most people learn the PowerShell pipeline by analogy: it's a Unix pipe, except the text is fancier. That analogy survives about twenty minutes.
PowerShell's pipeline moves .NET objects, one at a time, as a stream. Nothing is flattened to text unless you specifically ask for it. Commands receive live instances with properties, methods, and type information attached. Once that clicks, three things stop being mysterious: parameter binding, the ForEach-Object versus foreach question, and why your carefully written function keeps emitting junk you never asked for.
Objects in flight, not strings on a wire
Compare two commands that look like cousins:
Get-Process | Where-Object { $_.CPU -gt 60 }
Get-Process | Out-String -Stream | Select-String 'chrome'
The first one filters on a numeric property. Where-Object never parses anything; it reads $_.CPU off an object and compares numbers. The second deliberately flattens everything to strings first, which is what you want when you're grepping console output and exactly what you don't want when you're processing data.
The other half of the mental model is that the pipeline is a stream. Get-ChildItem -Recurse | ForEach-Object { ... } begins handing you objects before the enumeration finishes. Assign the same command to a variable and you've materialized the whole array in memory first. That one difference decides whether a script peaks at 40 MB or 4 GB when it hits a file server with two million files on it.
Binding: by value first, then by property name
This is where the Unix analogy causes the most damage. When you pipe into a command, PowerShell tries parameter binding in a specific order:
- By value — does the incoming object's type match (or coerce to) a parameter marked
ValueFromPipeline? - By property name — if not, does the object have a property whose name matches a parameter marked
ValueFromPipelineByPropertyName?
Here's a function that leans on the second rule:
function Stop-StaleJob {
[CmdletBinding()]
param(
[Parameter(ValueFromPipelineByPropertyName, Mandatory)]
[string]$Name
)
process { "Recycling job: $Name" }
}
Get-Job | Stop-StaleJob
Get-Job emits objects with a Name property, so binding succeeds even though a job object is not a string. Convenient. Also fragile. Rename the property upstream, or feed in an object that has JobName instead of Name, and the binding quietly matches nothing. If the parameter is mandatory you at least get an error. If it isn't, you get a function that runs, does nothing, and exits with a success code.
The by-value path has its own trap: implicit conversion. Pipe a FileInfo into a parameter typed [string] and PowerShell happily converts it to the file name. Your function "works" until someone passes a path with a space in it and the downstream command splits on whitespace. Declare the type you actually want — [IO.FileInfo], [datetime], [guid] — and let binding fail loudly when the input is wrong.
The process block is where pipelining lives
A function with only a param block and a body runs once. A function with a process block runs once per incoming object. That distinction is the whole game.
function Measure-TotalSize {
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline, Mandatory)]
[IO.FileInfo]$File
)
begin { $count = 0; $bytes = 0 }
process { $count++; $bytes += $File.Length }
end { [pscustomobject]@{ Files = $count; Bytes = $bytes } }
}
Get-ChildItem -Path C:\Logs -Recurse -File | Measure-TotalSize
Note that state lives in begin, accumulates in process, and reports in end. The common failure is putting the accumulation in begin and reading $input — that pattern works only when the function is called without the pipeline machinery involved, and it breaks the moment someone pipes something in.
ForEach-Object versus foreach
They are not the same command with different spelling.
foreach is a language statement. It enumerates a collection that already exists in memory, it's noticeably faster for in-memory work, and break and continue behave the way you expect.
ForEach-Object is a cmdlet. It accepts pipeline input one object at a time, which means it streams, and it exposes -Parallel in PowerShell 7. But break inside it does not stop the pipeline the way people assume, and continue behaves differently too.
Before — loads everything, then iterates:
$lines = Get-Content -Path D:\logs\huge.log
foreach ($line in $lines) { if ($line -match 'ERROR') { $hits++ } }
After — streams, constant memory:
Get-Content -Path D:\logs\huge.log -ReadCount 1000 |
ForEach-Object { foreach ($line in $_) { if ($line -match 'ERROR') { $hits++ } } }
Or better still for a genuinely huge file, [System.IO.File]::ReadLines() wrapped in a foreach. Rule of thumb: foreach when the data is already in memory, ForEach-Object when it isn't.
Output pollution: everything is a return value
In PowerShell, every expression that isn't captured goes to the output stream. That includes the accidental ones.
function Get-UserCount {
$bag = [System.Collections.ArrayList]::new()
$bag.Add('alice') # emits 0
$bag.Add('bob') # emits 1
return $bag.Count # emits 2
}
Call it and you get 0 1 2 instead of 2. The fix is to discard intentionally:
$null = $bag.Add('alice')
[void]$bag.Add('bob')
Write-Output $bag.Count
The same leak shows up with Write-Host used as a logging channel (it bypasses the pipeline entirely, so it can't be captured, redirected, or tested), with methods that return values you ignored, and with Out-Null sprinkled everywhere — which is fine for a single line but measurably slow in a tight loop.
If you want diagnostics that flow with the object stream, use Write-Verbose or Write-Information. If you want data, emit objects. Mixing the two is how a function that was supposed to return a list ends up returning a list plus four status messages.
The short version
Pipes carry objects, one at a time. Binding tries value first, property name second, and coercion can hide a bug. process blocks are the pipeline-aware units of work. foreach and ForEach-Object solve different problems. Every stray expression is a return value. Get those five things right and the shell stops feeling quirky and starts feeling deliberate — because it is.