Windows & Git Bash Compatibility for Azure Data Factory

Overview

Azure Data Factory development frequently occurs on Windows machines using Git Bash (MINGW64) as the primary shell. This introduces path conversion challenges that can break CI/CD pipelines, npm commands, and deployment scripts.

Git Bash Path Conversion Behavior

Automatic Path Conversion

Git Bash (MINGW) automatically converts Unix-style paths to Windows paths:

Conversions:

• /foo → C:/Program Files/Git/usr/foo

• /foo:/bar → C:\msys64\foo;C:\msys64\bar (path lists)

• --dir=/foo → --dir=C:/msys64/foo (arguments)

What Triggers Conversion:

• Leading forward slash (/ ) in arguments

• Colon-separated path lists

• Arguments after - or , with path components

What's Exempt:

• Arguments containing = (variable assignments)

• Drive specifiers (C: )

• Arguments with ; (already Windows format)

• Arguments starting with // (Windows switches)

ADF-Specific Path Issues

npm Build Commands

Problem:

This fails in Git Bash due to path conversion

npm run build validate ./adf-resources /subscriptions/abc/resourceGroups/rg/providers/Microsoft.DataFactory/factories/myFactory

Path gets converted incorrectly

Solution:

Disable path conversion before running

export MSYS_NO_PATHCONV=1 npm run build validate ./adf-resources /subscriptions/abc/resourceGroups/rg/providers/Microsoft.DataFactory/factories/myFactory

Or wrap the command

MSYS_NO_PATHCONV=1 npm run build export ./adf-resources /subscriptions/.../myFactory "ARMTemplate"

PowerShell Scripts

Problem:

Calling PowerShell scripts from Git Bash

pwsh ./PrePostDeploymentScript.Ver2.ps1 -armTemplate "./ARMTemplate/ARMTemplateForFactory.json"

Path conversion may interfere

Solution:

Disable conversion for PowerShell calls

MSYS_NO_PATHCONV=1 pwsh ./PrePostDeploymentScript.Ver2.ps1 -armTemplate "./ARMTemplate/ARMTemplateForFactory.json"

ARM Template Paths

Problem:

Azure CLI deployment from Git Bash

az deployment group create
--resource-group myRG
--template-file ARMTemplate/ARMTemplateForFactory.json # Path may get converted

Solution:

Use relative paths with ./ prefix or absolute Windows paths

export MSYS_NO_PATHCONV=1 az deployment group create
--resource-group myRG
--template-file ./ARMTemplate/ARMTemplateForFactory.json

Shell Detection Patterns

Bash Shell Detection

#!/usr/bin/env bash

Method 1: Check $MSYSTEM (Git Bash/MSYS2 specific)

if [ -n "$MSYSTEM" ]; then echo "Running in Git Bash/MinGW ($MSYSTEM)" export MSYS_NO_PATHCONV=1 fi

Method 2: Check uname -s (more portable)

case "$(uname -s)" in MINGW64*|MINGW32*|MSYS*) echo "Git Bash detected" export MSYS_NO_PATHCONV=1 ;; Linux*) if grep -q Microsoft /proc/version 2>/dev/null; then echo "WSL detected" else echo "Native Linux" fi ;; Darwin*) echo "macOS" ;; esac

Method 3: Check $OSTYPE (bash-specific)

case "$OSTYPE" in msys*) echo "Git Bash/MSYS" export MSYS_NO_PATHCONV=1 ;; linux-gnu*) echo "Linux" ;; darwin*) echo "macOS" ;; esac

Node.js Shell Detection

// detect-shell.js - For use in npm scripts or Node tools function detectShell() { const env = process.env;

// Git Bash/MinGW (MOST RELIABLE) if (env.MSYSTEM) { return { type: 'mingw', subsystem: env.MSYSTEM, // MINGW64, MINGW32, or MSYS needsPathFix: true }; }

// WSL if (env.WSL_DISTRO_NAME) { return { type: 'wsl', distro: env.WSL_DISTRO_NAME, needsPathFix: false }; }

// PowerShell (3+ paths in PSModulePath) if (env.PSModulePath?.split(';').length >= 3) { return { type: 'powershell', needsPathFix: false }; }

// CMD if (process.platform === 'win32' && env.PROMPT === '$P$G') { return { type: 'cmd', needsPathFix: false }; }

// Cygwin if (env.TERM === 'cygwin') { return { type: 'cygwin', needsPathFix: true }; }

// Unix shells if (env.SHELL?.includes('bash')) { return { type: 'bash', needsPathFix: false }; } if (env.SHELL?.includes('zsh')) { return { type: 'zsh', needsPathFix: false }; }

return { type: 'unknown', platform: process.platform, needsPathFix: false }; }

// Usage const shell = detectShell(); console.log(Detected shell: ${shell.type});

if (shell.needsPathFix) { process.env.MSYS_NO_PATHCONV = '1'; console.log('Path conversion disabled for Git Bash compatibility'); }

module.exports = { detectShell };

PowerShell Detection

Detect PowerShell edition and version

function Get-ShellInfo { $info = @{ Edition = $PSVersionTable.PSEdition Version = $PSVersionTable.PSVersion OS = $PSVersionTable.OS Platform = $PSVersionTable.Platform }

if ($info.Edition -eq 'Core') { Write-Host "PowerShell Core (pwsh) - Cross-platform compatible" -ForegroundColor Green $info.CrossPlatform = $true } else { Write-Host "Windows PowerShell - Windows only" -ForegroundColor Yellow $info.CrossPlatform = $false }

return $info }

$shellInfo = Get-ShellInfo

CI/CD Pipeline Patterns

Local Development Scripts

validate-adf.sh (Git Bash compatible):

#!/usr/bin/env bash set -e

Detect and handle Git Bash

if [ -n "$MSYSTEM" ]; then export MSYS_NO_PATHCONV=1 echo "🔧 Git Bash detected - path conversion disabled" fi

Configuration

ADF_ROOT="./adf-resources" FACTORY_ID="/subscriptions/${AZURE_SUBSCRIPTION_ID}/resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.DataFactory/factories/${FACTORY_NAME}"

Validate ADF resources

echo "📋 Validating ADF resources..." npm run build validate "$ADF_ROOT" "$FACTORY_ID"

Generate ARM templates

echo "📦 Generating ARM templates..." npm run build export "$ADF_ROOT" "$FACTORY_ID" "ARMTemplate"

echo "✅ Validation complete"

deploy-adf.sh (Cross-platform):

#!/usr/bin/env bash set -e

Detect shell

detect_shell() { if [ -n "$MSYSTEM" ]; then echo "git-bash" elif [ -n "$WSL_DISTRO_NAME" ]; then echo "wsl" elif [[ "$OSTYPE" == "darwin"* ]]; then echo "macos" else echo "linux" fi }

SHELL_TYPE=$(detect_shell) echo "🖥️ Detected shell: $SHELL_TYPE"

Handle Git Bash

if [ "$SHELL_TYPE" = "git-bash" ]; then export MSYS_NO_PATHCONV=1 fi

Download PrePostDeploymentScript

curl -sLo PrePostDeploymentScript.Ver2.ps1
https://raw.githubusercontent.com/Azure/Azure-DataFactory/main/SamplesV2/ContinuousIntegrationAndDelivery/PrePostDeploymentScript.Ver2.ps1

Stop triggers

echo "⏸️ Stopping triggers..." MSYS_NO_PATHCONV=1 pwsh ./PrePostDeploymentScript.Ver2.ps1
-armTemplate "./ARMTemplate/ARMTemplateForFactory.json"
-ResourceGroupName "$RESOURCE_GROUP"
-DataFactoryName "$FACTORY_NAME"
-predeployment $true
-deleteDeployment $false

Deploy ARM template

echo "🚀 Deploying ARM template..." az deployment group create
--resource-group "$RESOURCE_GROUP"
--template-file ./ARMTemplate/ARMTemplateForFactory.json
--parameters ./ARMTemplate/ARMTemplateParametersForFactory.json
--parameters factoryName="$FACTORY_NAME"

Start triggers

echo "▶️ Starting triggers..." MSYS_NO_PATHCONV=1 pwsh ./PrePostDeploymentScript.Ver2.ps1
-armTemplate "./ARMTemplate/ARMTemplateForFactory.json"
-ResourceGroupName "$RESOURCE_GROUP"
-DataFactoryName "$FACTORY_NAME"
-predeployment $false
-deleteDeployment $true

echo "✅ Deployment complete"

package.json with Shell Detection

{ "scripts": { "prevalidate": "node scripts/detect-shell.js", "validate": "node node_modules/@microsoft/azure-data-factory-utilities/lib/index validate", "prebuild": "node scripts/detect-shell.js", "build": "node node_modules/@microsoft/azure-data-factory-utilities/lib/index export" }, "dependencies": { "@microsoft/azure-data-factory-utilities": "^1.0.3" } }

scripts/detect-shell.js:

const detectShell = () => { if (process.env.MSYSTEM) { console.log('🔧 Git Bash detected - disabling path conversion'); process.env.MSYS_NO_PATHCONV = '1'; return 'git-bash'; } console.log(🖥️ Shell: ${process.platform}); return process.platform; };

detectShell();

Common Issues and Solutions

Issue 1: npm build validate fails with "Resource not found"

Symptom:

npm run build validate ./adf-resources /subscriptions/abc/...

Error: Resource '/subscriptions/C:/Program Files/Git/subscriptions/abc/...' not found

Cause: Git Bash converted the factory ID path

Solution:

export MSYS_NO_PATHCONV=1 npm run build validate ./adf-resources /subscriptions/abc/...

Issue 2: PowerShell script paths incorrect

Symptom:

pwsh PrePostDeploymentScript.Ver2.ps1 -armTemplate "./ARM/template.json"

Error: Cannot find path 'C:/Program Files/Git/ARM/template.json'

Cause: Git Bash converted the ARM template path

Solution:

MSYS_NO_PATHCONV=1 pwsh PrePostDeploymentScript.Ver2.ps1 -armTemplate "./ARM/template.json"

Issue 3: Azure CLI template-file parameter fails

Symptom:

az deployment group create --template-file ./ARMTemplate/file.json

Error: Template file not found

Cause: Path conversion interfering with Azure CLI

Solution:

export MSYS_NO_PATHCONV=1 az deployment group create --template-file ./ARMTemplate/file.json

Best Practices

1. Set MSYS_NO_PATHCONV in .bashrc

Add to ~/.bashrc for Git Bash

if [ -n "$MSYSTEM" ]; then export MSYS_NO_PATHCONV=1 fi

2. Create Wrapper Scripts

adf-cli.sh - Wrapper for ADF npm commands

#!/usr/bin/env bash export MSYS_NO_PATHCONV=1 npm run build "$@"

3. Use Relative Paths with ./

Prefer this (less likely to trigger conversion)

./ARMTemplate/ARMTemplateForFactory.json

Over this

ARMTemplate/ARMTemplateForFactory.json

4. Document Shell Requirements

README.md

Development Environment

Windows Users

• Use Git Bash or PowerShell Core (pwsh)
• Git Bash users: Add export MSYS_NO_PATHCONV=1 to .bashrc
• Alternative: Use WSL2 for native Linux environment

5. Test on Multiple Shells

Test matrix for Windows developers

• Git Bash (MINGW64)
• PowerShell Core 7+
• WSL2 (Ubuntu/Debian)
• cmd.exe (if applicable)

Quick Reference

Environment Variable Purpose Value

MSYS_NO_PATHCONV

Disable all path conversion (Git for Windows) 1

MSYS2_ARG_CONV_EXCL

Exclude specific arguments from conversion (MSYS2) * or patterns

MSYSTEM

Current MSYS subsystem MINGW64 , MINGW32 , MSYS

WSL_DISTRO_NAME

WSL distribution name Ubuntu , Debian , etc.

Resources

• Git for Windows Path Conversion

• MSYS2 Path Conversion

• Azure CLI on Windows

• PowerShell Core

Summary

Key Takeaways:

• Git Bash automatically converts Unix-style paths to Windows paths

• Use export MSYS_NO_PATHCONV=1 to disable conversion

• Detect shell environment using $MSYSTEM variable

• Test CI/CD scripts on all shells used by your team

• Use PowerShell Core (pwsh) for cross-platform scripts

• Add shell detection to local development scripts