Succeeding with PowerShell Universal Git Sync

PowerShell Universal

February 15, 2023

quote Discuss this Article

What is git sync?

Git synchronization (git sync) is a feature of PowerShell Universal. It creates commits based on changes within the PowerShell Universal configuration repository and pushes and pulls them from a git remote. Git sync provides configuration consistency across a high-availability PowerShell Universal cluster or can move changes from a dev environment to a production one through pull requests and branching. Even in a single-node environment, it’s also useful for keeping history of changes.

How does git sync work?

Git sync works by issuing git pull and git push commands on a regular interval to ensure that configuration files stay up to date with what is in the remote git repository. By default, every minute, the git sync process will run to pull changes from the git remote. When it comes to user interaction, git sync operates in two different modes.

Manual Mode

The default mode is Manual mode. In this mode, the PowerShell Universal server is effectively readonly until a user clicks the edit button in the admin console. The user can then make changes throughout the admin console. Once the changes are complete, the user can provide a commit message and push the changes to the remote and the server will again become readonly. While the user is editing the configuration files, the git sync process will not pull from the remote.

Note that the manual mode process changes the state of the entire server. If multiple administrators are making changes to the server at this time, they should be aware that their changes are not isolated to their account.

Automatic Mode

The second mode is Automatic mode. In this mode, the PowerShell Universal server is writable and any changes are committed and directly pushed to the git remote. In this mode, the user does not have the ability to provide a commit message but doesn’t have to worry about moving in and out of edit mode.

Git Sync Options

Aside from how the user interacts with git, the git sync process can also be configured to interact with a git remote in three different ways.

Use Database

The Use Database setting is used for storing the git repository directly in the PSU database. This can simplify the git sync process by avoiding having to setup a git remote like GitHub or BitBucket and deal with networking and credentials. The use database setting requires the git.exe command line tool, or a platform specific git version, is installed on the PowerShell Universal system. The use database setting takes advantage of git bundle to create and store bundles directly in the database. The disadvantage of this type of configuration is that the git repository is not easily accessible to third party tools.

You can learn more about git bundle here.

Internal Git Client

PowerShell Universal includes an internal git client that can push and pull from remote repositories. This client, libgit2sharp, wraps a C-based git client library. It does not provide the full functionality of git.exe but requires no git client installation on the PSU server. You can include credentials in PowerShell Universal’s git settings that will be used with the git client library.

Because the internal git client library requires specific implementations in the PSU server, it is limited in functionality from the external git client. That said, for most users, it should suffice. Some limitations include:

To configure the internal git client, uncheck the Use Database setting and insert your git remote URL and credentials.

External Git Client

PowerShell Universal also integrates with the external git client command line process. Git commands such as git push, git pull and git commit are directly invoked via an external git.exe process. When using external git client integration, PowerShell Universal will not pass credentials to the git commands, so you are required to specify them in another manner.

You can specify credentials in the following ways:

URL

You can use a username and password (or personal access token) directly in the URL with the following syntax. This format requires no additional git configuration but does store the URL in plaintext within the database.

https://adam:PAT@github.com/ironmansoftware/shiny-octowaffle.git

Git Credential Helper

You can also use a git credential helper to store credentials for specific repositories within git. PowerShell Universal does not integrate directly with credential helpers but you can configure them outside of the PSU environment, and when the git sync process is run, the credentials will be used by git.exe itself.

You can learn more about git credentials here. Note that any git configuration settings that are user specific need to be made in the PSU service account’s profile.

SSH Private Key

Similar to the git credential helper, you can also store a git SSH private key for the PSU git client use. Most git repositories allow for uploading custom public keys for accessing git repos rather than with credentials.

As an example, you can follow the SSH configuration documentation for GitHub here.

Additional Settings

In addition to credentials, any changes made to either the user-local or global git settings will be loaded by the PowerShell Universal git sync process. These settings can include options like SSL-crypto backend, line endings and whitespace.

Settings up Git Sync

Git sync is setup by configuring the git settings in either the appsettings.json file or by using the Git Settings dialog under Settings \ Git. There are several configuration options that you need to be aware of when settings up git sync. We recommend the database, when possible. It makes it easier to configure multiple nodes to use the same git settings when they share a database.

Credential Information

Most git repositories employ some sort of personal access token. Personal access tokens can be used as passwords, and treated as such, but can also be limited in scope and revoke from the server when necessary. This is our preferred method for credentials.

It’s also possible to use your username and password directly.

First Sync

First, you need to be aware of the first push or pull from a git remote. Note that although there is a init behavior, this setting is no longer required to be set and can be ignored. It will be removed in a future version.

Files Local

If you have not configured git sync before, this is the most common scenario. You have local configuration files that you wish to store in a git remote. The git remote needs to be bare. There should be no branches created but the repository should exist. In this scenario, PowerShell Universal will initialize a local git repository, create the branch specified and then initialize the remote via a git push.

After the first sync, the local files will exist in the remote.

Files in Remote

If you have a git remote that already has a branch created with files in it, you have to ensure that your PowerShell Universal configuration repository is empty on the local server. If files exist, the first git sync will fail. If the directory doesn’t exist, a git clone on the specified branch will be called and the files will be pulled locally. PowerShell Universal will then reconfigure based on the files pulled.

Git Sync Interval

By default, the git sync process runs every minute to check for changes that may have been pushed to the git remote. A git pull is executed and then the current HEAD commit of the branch is compared to the last git sync commit SHA. If the HEAD commit SHA is different than the previous commit SHA, the PowerShell Universal configuration system will reload changed files.

Sync Behavior

There are three sync behaviors that exist within PowerShell Universal. These change how PowerShell Universal interacts with the remote git repository.

Two-Way

In two-way mode, a git pull and git push are issued during each git sync. The PowerShell Universal instance is read-write.

One-Way

In one-way mode, a git pull is issued during each git sync. The PowerShell Universal instance is read-only. This is useful for production environments where changes are coming via pull requests and not made directly to the PowerShell Universal server.

Push-Only

In push-only mode, a git push is issued if changes exist during each git sync. The PowerShell Universal instance is read-write but will not pull changes from the remote. This is useful when editing files in a debug environment where you do not wish to deal with merge conflicts from changes coming from the remote.

Common Issues

Below are some common issues you may run into when dealing with git sync.

Merge Conflicts

PowerShell Universal does not handle merge conflicts and they require manual intervention. Because of this, it may be desired to have production environments in one-way git sync mode to avoid local changes. All changes and merges will be done using the git tools available in the git repository or other local development environments.

If you wish to have a read\write production environment with multiple nodes, it would be recommend to setup multiple one-way nodes and a single two-way node.

If you do encounter a merge conflict, you will need to edit the files directly on the PSU server in order to fix the conflict.

Authentication Issues

Every git remote server is a bit different. For example, BitBucket has several levels of HTTP tokens and you need to ensure they are properly configured in order to use them against your repositories. We recommend ensuring that you can clone repositories with a standard git client before attempting to use a token within PowerShell Universal. If you do wish to test credential within PowerShell Universal, you can take advantage of the Synchronize Now button on the Git page to verify access without having to wait for the scheduled git sync.

SSL Issues

The internal git client does not allow for acceptance of untrusted certificates. If your server is experiencing issues with SSL connections, you may need to configure an external git client and setup the appropriate git config settings to avoid errors and accept any insecurities that may result from these changes.

Git Hangs

By default, the PowerShell Universal git sync process will wait 60 seconds for the git.exe process to terminate. In certain systems, git can spawn a UI for credential purposes. This can cause git sync to hang and fail.