Ironman Software Forums
Continue the conversion on the Ironman Software forums. Chat with over 1000 users about PowerShell, PowerShell Universal, and PowerShell Pro Tools.
Discuss this Article
PowerShell provides a great deal of functionality and much of it is hidden behind .NET cmdlets that help abstract away complex interactions that typical scripters don’t need to know about. When scripts start sucking memory and CPU, it can be hard to diagnosis what may be happening. Tools like PSProfiler and forcing GCCollect are sometimes answers to these questions, they typically don’t reveal the root cause of issues.
Enter the RuntimeDiagnostics module. This module is built on the Microsoft.Diagnostics.Runtime library and is used to debug process threads and memory usage. You can see full stack traces and object allocations within any .NET 6+ process (like pwsh.exe).
Installation
You can install this module from the PowerShell Gallery with Install-Module or Install-PSResource.
Install-PSResource RuntimeDiagnostics
Getting Started
.NET processes can be debugged in two different ways: live and postmortem. To debug a live process, look at the Debug-ClrProcess cmdlet.
$Process = Get-Process pwsh | Debug-ClrProcess
Once you have a process, you can mount the CLR runtime and start interrogating the internals of PowerShell.
$Process | Mount-ClrRuntime | Get-ClrThreads | Show-ClrStackTrace
If you want to capture a process and interrogate it offline, you can collect a memory dump. Tools like dotnet-dump provide this functionality.
dotnet-dump collect $ProcessId
You’ll be able to load the memory dump on any machine using the Import-ClrMemoryDump cmdlet.
Import-ClrMemoryDump -Path .\dump.dmp | Mount-ClrRuntime | Get-ClrThreads | Show-ClrStackTrace
Common Scenarios
Below are some common scenarios you may want to use RuntimeDiagnostics for.
Memory Leaks
.NET is a managed memory platform but it does not totally prevent it from experiencing memory leaks. PowerShell scripts, especially in multi-runspace or long running runspace environments like PowerShell Universal, can start to leak memory over time. Often this is caused by modules that aren’t necessarily designed to work in such ways.
To identify a memory leak, you can use the Get-ClrObject -Statistics command to list out the largest object types within the process.
First, let’s create a really large variable. While a trivial example, this snippet creates a very large string. While this script runs (it will take awhile) you’ll see memory climb.
$SB = [System.Text.StringBuilder]::new()
for($i = 0; $i -lt 10000000; $i++) { $null = $sb.Append("x") }
$S = $SB.ToString()
After running this script, my PowerShell process is now sitting at 150 MB of memory.
If we run the Debug-ClrProcess cmdlet and get the object stats, you’ll see that strings take up the largest about of memory.
Get-Process -Id 1080 | Debug-ClrProcess | Mount-ClrRuntime | Get-ClrObject -Statistics | Select -First 10
Type Count Size
---- ----- ----
System.String 43044 44316866
System.Char[] 8574 39625254
System.Management.Automation.VariableScopeItemSearcher 23391 1684152
System.Management.Automation.SessionStateScopeEnumerator 23544 753408
System.Object[] 7767 621864
System.Text.StringBuilder 8490 407520
System.Int32[] 3605 352472
System.Reflection.RuntimeMethodInfo 2796 290784
Free 1566 285936
System.Management.Automation.VariablePath 6599 263960
As expected, strings take up most of the memory. We can find the largest string with the following command.
Get-Process -Id 1080 | Debug-ClrProcess | Mount-ClrRuntime | Get-ClrObject -Type 'System.String' -Largest 1
To look at the content of the string, use Show-ClrObject.
Get-Process -Id 1080 | Debug-ClrProcess | Mount-ClrRuntime | Get-ClrObject -Type 'System.String' -Largest 1 | Show-ClrObject
This example isn’t too interesting as it just displays a large screen of xs.
CPU Usage
Another common scenario is high CPU usage by scripts. Profiler modules may help in this scenario but using RuntimeDiagnostics you can get to the root of the issue.
In this example, we’ll use the dotnet-dump global tool. You can learn more about it here.
First, we’ll start a script that takes a long time. ConvertTo-Json is notorious for causing unexpected performance issues. This script will run for a few seconds.
Get-Process | ConvertTo-Json
While the script is running, in another process, we can collect a memory dump with the following.
dotnet-dump collect -p 1080 -o .\dump.dmp
The resulting dmp file can be loaded by the RuntimeDiagnostics module. To do so, use the Import-ClrMemoryDump cmdlet.
$Dump = Import-ClrMemoryDump -Path .\dump.dmp
Next, we’ll look at the threads that are currently running and print out the stack traces.
$Dump | Mount-ClrRuntime | Get-ClrThread | Show-ClrStackTrace
In my lab, thread 20 is running the pipeline. You’ll notice that the pipeline is calling the ConvertToJsonCommand class. While it may look like a bunch of gobblygook, the below is a .NET stack trace that includes all the methods called by the ConvertTo-Json cmdlet. The more you look at stack traces like this, the more you’ll see the girl in red.
Managed thread ID: 20
System.Threading.Monitor.ObjWait(Int32, System.Object)
System.Threading.ManualResetEventSlim.Wait(Int32, System.Threading.CancellationToken)
Microsoft.Management.Infrastructure.CimCmdlets.CimAsyncOperation.ProcessRemainActions(Microsoft.Management.Infrastructure.CimCmdlets.CmdletOperationBase)
System.Management.Automation.CommandProcessorBase.Complete()
System.Management.Automation.CommandProcessorBase.DoComplete()
System.Management.Automation.Internal.PipelineProcessor.DoCompleteCore(System.Management.Automation.CommandProcessorBase)
System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(System.Object)
System.Management.Automation.PipelineOps.InvokePipeline(System.Object, Boolean, System.Management.Automation.CommandParameterInternal[][], System.Management.Automation.Language.CommandBaseAst[], System.Management.Automation.CommandRedirection[][], System.Management.Automation.Language.FunctionContext)
DynamicClass.<ScriptBlock>(System.Runtime.CompilerServices.Closure, System.Management.Automation.Language.FunctionContext)
System.Management.Automation.ScriptBlock.InvokeWithPipeImpl(System.Management.Automation.ScriptBlockClauseToInvoke, Boolean, System.Collections.Generic.Dictionary`2<System.String,System.Management.Automation.ScriptBlock>, System.Collections.Generic.List`1<System.Management.Automation.PSVariable>, ErrorHandlingBehavior, System.Object, System.Object, System.Object, System.Management.Automation.Internal.Pipe, System.Management.Automation.InvocationInfo, System.Object[])
System.Management.Automation.ScriptBlock.InvokeWithPipe(Boolean, ErrorHandlingBehavior, System.Object, System.Object, System.Object, System.Management.Automation.Internal.Pipe, System.Management.Automation.InvocationInfo, Boolean, System.Collections.Generic.List`1<System.Management.Automation.PSVariable>, System.Collections.Generic.Dictionary`2<System.String,System.Management.Automation.ScriptBlock>, System.Object[])
System.Management.Automation.PSScriptProperty.InvokeGetter(System.Object)
Microsoft.PowerShell.Commands.JsonObject.AppendPsProperties(System.Management.Automation.PSObject, System.Collections.IDictionary, Int32, Boolean, ConvertToJsonContext ByRef)
Microsoft.PowerShell.Commands.JsonObject.AddPsProperties(System.Object, System.Object, Int32, Boolean, Boolean, ConvertToJsonContext ByRef)
Microsoft.PowerShell.Commands.JsonObject.ProcessValue(System.Object, Int32, ConvertToJsonContext ByRef)
Microsoft.PowerShell.Commands.JsonObject.ProcessEnumerable(System.Collections.IEnumerable, Int32, ConvertToJsonContext ByRef)
Microsoft.PowerShell.Commands.JsonObject.ProcessValue(System.Object, Int32, ConvertToJsonContext ByRef)
Microsoft.PowerShell.Commands.JsonObject.ConvertToJson(System.Object, ConvertToJsonContext ByRef)
Microsoft.PowerShell.Commands.ConvertToJsonCommand.EndProcessing()
System.Management.Automation.CommandProcessorBase.Complete()
System.Management.Automation.CommandProcessorBase.DoComplete()
System.Management.Automation.Internal.PipelineProcessor.DoCompleteCore(System.Management.Automation.CommandProcessorBase)
System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(System.Object)
System.Management.Automation.PipelineOps.InvokePipeline(System.Object, Boolean, System.Management.Automation.CommandParameterInternal[][], System.Management.Automation.Language.CommandBaseAst[], System.Management.Automation.CommandRedirection[][], System.Management.Automation.Language.FunctionContext)
System.Management.Automation.Interpreter.ActionCallInstruction`6[[System.__Canon, System.Private.CoreLib],[System.Boolean, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].Run(System.Management.Automation.Interpreter.InterpretedFrame)
System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(System.Management.Automation.Interpreter.InterpretedFrame)
System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(System.Management.Automation.Interpreter.InterpretedFrame)
System.Management.Automation.Interpreter.Interpreter.Run(System.Management.Automation.Interpreter.InterpretedFrame)
System.Management.Automation.Interpreter.LightLambda.RunVoid1[[System.__Canon, System.Private.CoreLib]](System.__Canon)
System.Management.Automation.DlrScriptCommandProcessor.RunClause(System.Action`1<System.Management.Automation.Language.FunctionContext>, System.Object, System.Object)
System.Management.Automation.DlrScriptCommandProcessor.Complete()
System.Management.Automation.CommandProcessorBase.DoComplete()
System.Management.Automation.Internal.PipelineProcessor.DoCompleteCore(System.Management.Automation.CommandProcessorBase)
System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(System.Object)
System.Management.Automation.Runspaces.LocalPipeline.InvokeHelper()
System.Management.Automation.Runspaces.LocalPipeline.InvokeThreadProc()
System.Management.Automation.Runspaces.PipelineThread.WorkerProc()
System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
When debugging CPU issues like this, it’s good practice to take a couple of memory dumps to try to help identify the issue. Since the memory dump is just a snapshot in time, it may indicate that something that is running is the culprit when it is actually not the real problem. If you run multiple memory dumps and all of them have the same stack trace, then it’s likely you’ve identified what is causing the issue.
Conclusion
In this post, we looked at how to use the RuntimeDiagnostics module to identify problems with PowerShell scripts. While this post was focused on PowerShell, there is nothing stopping you from using the module against any .NET application.
We’ll be recommending techniques in this post to PowerShell Universal users that are experiencing issues with certain modules. It requires no paid tools and can be run pretty quickly against most environments.
Feel free to open issues and pull requests on the RuntimeDiagnostics GitHub repository.