Microsoft Azure has a great set of capabilities for managing non-Azure based servers, including monitoring, policy evaluation, inventory and change tracking, and security tools. Access to those services for non-Azure servers may be via Azure Arc – specifically installation of the Azure Connected Machine Agent. Let’s dig into the technical specifications of the agent and how to take care of it.
Overview of the Azure Connected Machine Agent
Architecture
The Azure Connected Machine Agent:
-
Creates an identity in Azure for the server (with an Azure Resource ID) under Azure Arc-enabled Servers
- Manages the machine’s connection to Azure, with the Hybrid Instance Metadata Service
- Handles a guest configuration agent for policy assessment
- And runs an extension agent to enable specific post-deployment configuration and automation tasks (for example, the Custom script extension or Azure Key Vault Certificate Sync). For a list of current extensions visit Virtual machine extensions management with Azure Arc-enabled servers.
Architecture of the Azure Connected Machine Agent
In essence, this identity and connectivity allows your server to now be seen and controlled by some important Azure services.
Files, folders, services and accounts
NB: This information is accurate at the time of writing but may change in later versions of the agent.
The AzureConnectedMachineAgent.msi file that is downloaded by the deployment script onto the non-Azure server, is approx. 70 MB in size.
Installing the agent creates:
AzureConnectedMachineAgent folder under %ProgramFiles% - approx. 220 MB by default
%SYSTEMDRIVE%\packages folder of 225 MB for extension package executables (including Microsoft.Azure.AzureDefenderForServers.MDE.Windows, Microsoft.Azure.Monitoring.DependencyAgent.DependencyAgentWindows, Microsoft.Azure.Security.IaaSAntimalware and Microsoft.EnterpriseCloud.Monitoring.MicrosoftMonitoringAgent
%ProgramData%\GuestConfig folder of 300MB and a %ProgramData%\AzureConnectedMachineAgent folder of 1MB, consisting of JSON configuration files, log files and the identity token for the Azure Connected Machine Agent CLI and the instance metadata service.
It also creates 3 services: the Azure Hybrid Instance Metadata Service (himds), the Guest Configuration Arc Service (gc_service) and the Guest configuration Extension Service (gc_service).
It creates an unprivileged account called NT SERVICE\himds used to run the Hybrid Instance Metadata Service, and a local security group Hybrid agent extension applications, which can request Azure Active Directory tokens for the system-assigned managed identity.
Finally, it adds the following environment variables:
IDENTITY_ENDPOINT http://localhost:40342/metadata/identity/oauth2/token
IMDS_ENDPOINT http://localhost:40342
System resource consumption
The agent is designed to have a minimal impact on system resources when idle, at approximately 0.07% CPU usage. When actively running, the Guest Configuration agent and Extension service agent can use up to 5% of the CPU, while extension components can peak higher. For details, visit Agent Resource Governance.
Security considerations
Because the Azure Connected Machine agent uses public key authentication to communicate with Azure Services, a private key will be stored on the local server’s disk. We recommend using full disk encryption, like BitLocker, to guard against key theft.
You can also configure the agent to limit which extensions are allow to be installed on your server and you can disable Guest Configuration. For information on locked down machine best practices and the Azure managed identity used for Arc-enabled servers, visit Security overview.
Metadata about your connected machine is collected and registered in Azure, within the region that your Arc-enabled servers identity is registered. This includes the computer name, domain name, BIOS ID, and some cloud provider metadata like Instance ID if the server is running in AWS or GCP. For a full list of the instance metadata that is collected, visit the Connected Machine agent overview.
Networking
By default, Azure Arc communicates on public IP addresses. For a list of current address prefixes used by Azure (including region-specific prefixes), download the Azure IP Ranges and Service Tags (Public Cloud) and search for AzureArcInfrastructure and the Azure services you will be using.
If you are required to keep communication between your systems and Microsoft Azure off public networks, you can utilize Azure Private Link, to send data over Azure ExpressRoute of a site-to-site VPN connection. For Private Link considerations and configuration, visit Use Azure Private Link to securely connect servers to Azure Arc.
Deployment methods
The Azure Connected Machine agent can be deployed via:
- Manually running a deployment script on the target machine
- Remotely via the Windows Admin Center
- Locally or remotely via PowerShell
- Remotely via the Windows PowerShell Desired State Configuration
- Remotely via Configuration Manager (PowerShell script or custom task sequence)
- Remotely via Group Policy
Learn more about these options at Azure Connected Machine agent deployment options.
Upgrading the agent
Verifying the installed version number
You can find the version number of the Azure Connected Machine Agent for a specific server on the Overview page for that machine under Azure Arc | Servers:
Azure Arc for Servers Overview page shows agent version number currenlty installed
Or you can use the azcmagent command on the server to list the agent version number (azcmagent version).
At scale
Azure Advisor will also surface recommendations of Arc-enabled servers that are running a previous version of the agent.
You can run a query against the Azure Resource Graph to find Arc-enabled servers with outdated agents, using KQL (Kusto), the Azure CLI or Azure PowerShell. For the command syntax, visit List Arc-enabled servers not running latest released agent version.
Upgrade methods
At the time of writing, the Azure Connected Machine Agent does not automatically update itself, but it’s on the product roadmap to add initiating an upgrade from Azure (either manually or automatically). Until that is released, there are several options for upgrading the agent:
Configure Microsoft Update to check for other Microsoft product updates. This will include the Azure Connected Machine agent. This can also be utilized if you’re using Azure for your update management.
Download and manually install the latest agent using the setup wizard from https://aka.ms/AzureConnectedMachineAgent including upgrading silently from the command line with msiexec.
msiexec.exe /i AzureConnectedMachineAgent.msi /qn /l*v "C:\Support\Logs\azcmagentupgradesetup.log"
Troubleshooting
The Azure portal’s Azure Arc | Servers page will show you your Arc-enabled servers at a glance, including if they are currently connected to Azure or they are offline. A server can be offline for a number of reasons: the server may be down, the agent may not be running, or networking connectivity between the server and Azure may be compromised.
Status of Servers with the Azure Arc agent deployed
An Azur Arc-enabled server showing as Offline to Azure
In addition to your usually server troubleshooting methods, you can use the inbuilt azcmagent command with the show parameter to view details of the Azure Connected Machine Agent, including:
- the machine’s resource ID in Azure
- the location of the agent logfile
- the agent’s last successful heartbeat signal received by Azure
- any agent error details
- the status of the agent’s dependent services
Result of the azcmagent show command to check the status of the Azure Connected Machine Agent
azcmagent show reporting an error with the service
The azcmagent logs command can also be used to create a zip of the most recent or all agent log files.
If Azure Arc for Server is reporting that the machine is offline but you can access it via any other method, you can run azcmagent connect to attempt to re-establish the connection to Azure. For full details, visit Troubleshooting Azure Arc for Servers – Status Offline.
And to learn more about the agent error codes, verbose logging and known error with the connection to Azure, visit Troubleshoot Azure Connected Machine agent connection issues.
Learn more:
Overview of the Azure Connected Machine Agent
Managing and maintaining the Connected Machine Agent