Ironman Software Forums
Continue the conversion on the Ironman Software forums. Chat with over 1000 users about PowerShell, PowerShell Universal, and PowerShell Pro Tools.
Discuss this Article
With the recent open source release of PowerShell Pro Tools, we provided our packaging process to the community. This blog posts dives into the details of how this process works.
Basic Principles
The basic process of packaging a PowerShell script into an executable involves including the script in an executable and the running it with the PowerShell SDK. There are various ways to accomplish this but the general process is the same.
Basically, the executable needs to read the embedded PowerShell script and then run the following code.
return ConsoleShell.Start(RunspaceConfiguration.Create(), null, null, myArgs.ToArray());
The ConsoleShell class is part of the PowerShell SDK and the class that also runs at the start of pwsh.exe. It takes a runspace configuration and some arguments. PowerShell Pro Tools would pass in a command and some additional arguments to run the script.
myArgs.AddRange(new[] { "-Command", contents.TrimEnd('\r', '\n') });
myArgs.AddRange(arguments);
myArgs.AddRange(new[] { "-PoshToolsRoot", "\"" + AssemblyDirectory + "\""});
This also allowed the compiled executable to accept arguments and pass them to the PowerShell SDK.
static int Main(string[] args)
{
var arguments = new List<string>();
foreach (var arg in args)
{
var argument = arg;
if (arg.Contains(" "))
{
argument = $"'{arg}'";
}
arguments.Add(argument);
}
Once passed to PowerShell, it would take it from there. It would parse the script and run just like any other pwsh.exe process.
Developer Compilation Technique
The first technique that we employed uses the .NET SDK to compile executables and embed the script into the executable. This technique requires that the user has the .NET SDK installed on their machine. This is the technique we used for most of the life of PowerShell Pro Tools. The general idea was that PowerShell Pro Tools would generate a .csproj file, a combined script file, resources and then compile the executable.
This package process would run via Merge-Script, an MSBuild task in PowerShell Tools for Visual Studio, or with pspack.exe.
Before packaging the scripts were analyzed to determine what modules were used and if any other scripts were called by the root script. This bundling step would combine all the scripts into a single file and mark modules that needed to be included as resources. This requires a lot of AST parsing and code manipulation. This is the most difficult part. Some modules worked better than others when bundled.
// Bundle other scripts
var bareWordDotSourced = ast.FindAll(m => m is CommandAst, true).Cast<CommandAst>().Where(x => x.CommandElements.OfType<StringConstantExpressionAst>().Any(m => m.StringConstantType == StringConstantType.BareWord && m.Value.StartsWith(".") && m.Value.EndsWith(".ps1", StringComparison.OrdinalIgnoreCase)));
FindReplacements(root, bareWordDotSourced, bundleResult, replacements);
var expandableString = ast.FindAll(m => m is CommandAst, true).Cast<CommandAst>().Where(x => x.InvocationOperator == TokenKind.Dot && x.CommandElements.OfType<ExpandableStringExpressionAst>().Any(m => m.StringConstantType == StringConstantType.DoubleQuoted && m.Value.EndsWith(".ps1", StringComparison.OrdinalIgnoreCase)));
FindReplacements(root, expandableString, bundleResult, replacements);
var doubleQuotedString = ast.FindAll(m => m is CommandAst, true).Cast<CommandAst>().Where(x => x.InvocationOperator == TokenKind.Dot && x.CommandElements.OfType<StringConstantExpressionAst>().Any(m => (m.StringConstantType == StringConstantType.DoubleQuoted || m.StringConstantType == StringConstantType.SingleQuoted) && m.Value.EndsWith(".ps1", StringComparison.OrdinalIgnoreCase)));
FindReplacements(root, doubleQuotedString, bundleResult, replacements);
The bundling process also requires setting special variables, like $PSScriptRoot, to the correct value. During runtime, the executable would set these variables to the correct value since it wouldn’t normally exist at runtime.
if (expandableString != null)
{
if (expandableString.Value.ToLower().Contains("$psscriptroot"))
{
return expandableString.Value.ToLower().Replace("$psscriptroot", rootFileInfo.DirectoryName);
}
}
Pre-written host files were used to generate the actual C# code that would be updated and included in the executable. During execution, they would read the script from the resources and run it.
String script;
using (var stream = Assembly.GetExecutingAssembly().GetManifestResourceStream("//FileName.script.ps1"))
{
using (StreamReader reader = new StreamReader(stream))
{
script = reader.ReadToEnd();
}
}
The next step would be to write our a .csproj file based on the configured settings. Since MSBuild uses XML, we just used the standard XML classes in C# to generate this file.
if (config.PowerShellCore)
{
references.Add(new XElement("ItemGroup",
new XElement("PackageReference",
new XAttribute("Include", "Microsoft.PowerShell.SDK"),
new XElement("Version", config.PowerShellVersion))
));
}
else
{
references.Add(new XElement("ItemGroup",
new XElement("Reference",
new XAttribute("Include", "System.Windows.Forms"))
));
references.Add(new XElement("ItemGroup",
new XElement("Reference",
new XAttribute("Include", "Microsoft.PowerShell.ConsoleHost"),
new XElement("HintPath", @"C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.PowerShell.ConsoleHost\v4.0_3.0.0.0__31bf3856ad364e35\Microsoft.PowerShell.ConsoleHost.dll"))
));
references.Add(new XElement("ItemGroup",
new XElement("Reference",
new XAttribute("Include", "System.Management.Automation"),
new XElement("HintPath", @"C:\Windows\Microsoft.NET\assembly\GAC_MSIL\System.Management.Automation\v4.0_3.0.0.0__31bf3856ad364e35\System.Management.Automation.dll"))
));
}
Modules that were bundled were copied to a temporary directory and stored in a ZIP file. Once the ZIP file was created, it was included as a resource in the executable.
private string ZipModules(string stagingDirectory, StageResult previousStage, PackageConfig config)
Compilation was done with the dotnet command line tool. It ran dotnet publish to produce the executable. For PowerShell 7 executables, we would package as a single file. This produced very large (100mb+) executables because they contained the entire .NET runtime and PowerShell SDK.
result = StartDotNet($"publish -c Release --self-contained -r {config.RuntimeIdentifier} -p:PublishSingleFile=true -p:IncludeAllContentForSelfExtract=true -o \"{Path.Combine(tempDirectory, "out")}\"", tempDirectory);
After execution, if obfuscation was enabled, we would either run Obfuscar or modify the PE header for the executable to throw off decompilers. While not a perfect solution, it would throw off the casual observer. Obfuscation is never a perfect solution and can always be reversed by a determined attacker.
private void CorruptPE(string assemblyPath)
{
var data = File.ReadAllBytes(assemblyPath);
ReadOnlySpan<byte> bundleSignature = new byte[] {
// 32 bytes represent the bundle signature: SHA-256 for ".net core bundle"
0x8b, 0x12, 0x02, 0xb9, 0x6a, 0x61, 0x20, 0x38,
0x72, 0x7b, 0x93, 0x02, 0x14, 0xd7, 0xa0, 0x32,
0x13, 0xf5, 0xb9, 0xe6, 0xef, 0xae, 0x33, 0x18,
0xee, 0x3b, 0x2d, 0xce, 0x24, 0xb3, 0x6a, 0xae
};
for(var i = 0; i < data.Length; i++)
{
if (data[i] == 0x8b && bundleSignature.SequenceEqual(new ReadOnlySpan<byte>(data, i, bundleSignature.Length)))
{
data[i] = 0x00;
}
}
File.WriteAllBytes(assemblyPath, data);
}
Resource Update Technique
One problem with the above technique is that it was flagged by a lot of antivirus products. Even signing the executable with a code signing certificate didn’t always fool them. The general concept is a bit shady so it’s not surprising that it was flagged. In an effort to solve this, we developed a new technique that used pre-compiled executables that could read a script from a resource and execute it. It also didn’t require the .NET SDK to be installed on the machine.While the technique was different, the general process was the same. We still called the ConsoleShell class to run the script.
internal class DefaultConsoleExecutor : IExecutor
{
public int Run(string script, string[] args)
{
var arguments = new List<string>();
arguments.Add("-Command");
arguments.Add(script.TrimEnd(new[] { '\r', '\n' }));
arguments.AddRange(args);
#if NET472
var config = RunspaceConfiguration.Create();
config.InitializationScripts.Append(new ScriptConfigurationEntry("Init", $"$Global:ExecutableRoot = '{AssemblyDirectory}'"));
return ConsoleShell.Start(config, null, null, arguments.ToArray());
#else
return ConsoleShell.Start(InitialSessionState.CreateDefault(), null, null, arguments.ToArray());
#endif
}
The host file was very similar to the previous technique but instead of compiling the host manually, the host executable was actually updated with the UpdateResource function. A helpful library called ASMResolver solved most of the heavy lifting. This results in a very simple function to set resources by known IDs that can then be read by the executable.
void AddScript(IPEImage peImage, StageResult previousStage, PackageProcess process)
{
var scriptContents = File.ReadAllBytes(previousStage.OutputFileName);
if (scriptContents[0] == 239)
{
scriptContents = scriptContents.Skip(3).ToArray();
}
var scriptString = Encoding.UTF8.GetString(scriptContents);
scriptString = ProcessScript(scriptString, process.Config.Package);
var script = new ResourceData(id: 101, contents: new DataSegment(Encoding.UTF8.GetBytes(scriptString)));
peImage.Resources.GetDirectory(666).AddOrReplaceEntry(script);
}
When the executable runs, it just grabs the resource from the PE and runs it.
var resource = NativeResourceManager.GetResourceFromExecutable(101, 666);
if (resource[0] == 239)
{
resource = resource.Skip(3).ToArray();
}
var script = Encoding.UTF8.GetString(resource.ToArray()).Trim();
In the end, the binaries were still flagged by antivirus products relatively quickly. Loading PowerShell scripts from memory and running them via the PowerShell SDK is risky behavior.
Looking Forward and Conclusion
There were some additional features that never were implemented that would have improved the packaging process. First, developing more pre-compiled host executables would have reduced the need to compile the host executable on the fly. This simplifies the tooling but increases the overhead during PowerShell Tools development. It likely would require some hosting of the executables to prevent having to package every possible platform into a single install. Second, developing a method of locating the PowerShell 7 runtime on the machine would have reduced the size of the executables and improved startup time. This would require some sort of bootstrapper process that would locate the runtime and then run the executable.
We will continue to support this project on GitHub and look forward to feedback and contribution from the community.