Cost governance is the continuous process of implementing policies to control the costs of services you use in Azure. This document provides cost governance considerations and recommendations for you to keep in mind while using Azure Arc-enabled Kubernetes.

Azure Arc-enabled Kubernetes provides two types of services:

• Azure Arc control plane functionality, which is provided at no extra cost and includes:

  • Resource organization through Azure management groups and tags.
  • Searching and indexing through Azure Resource Graph.
  • Access control through Azure role-based access control (RBAC) at the subscription or resource group level.
  • Automation through templates and extensions.

• Azure services used together with Azure Arc-enabled Kubernetes incur costs according to their usage. These services include:

  • Kubernetes GitOps configuration
  • Azure Policy for Kubernetes
  • Azure Monitor Container Insights
  • Microsoft Defender for Containers
  • Microsoft Sentinel
  • Azure Key Vault

• Governance: Define a governance plan for your hybrid clusters that translates into Azure Policies, tags, naming standards and least-privilege controls.

• Azure Monitor Container Insights: Azure Monitor Container Insights provides telemetry visibility by collecting performance metrics from controllers, nodes, and containers available in Kubernetes through the Metrics API. Container logs are also collected. This is billed by data ingestion, retention, and exports.

• Microsoft Defender for Cloud: Microsoft Defender for Cloud is offered in two modes:

  Without enhanced security features (Free) - Microsoft Defender for Cloud is enabled for free on all your Azure subscriptions when you visit the workload protection dashboard in the Azure portal for the first time, or if you enable it programmatically via API. This free mode provides the secure score and its related features: security policy, continuous security assessment, and actionable security recommendations for your Azure resources.

  With all enhanced security features (Paid) - Enabling Microsoft Defender for Cloud enhanced security extends the capabilities of free mode to workloads running in private and other public clouds, providing unified security management and threat protection across your hybrid cloud workloads.

• Kubernetes GitOps configuration: Kubernetes GitOps configuration delivers configuration management and application deployment using GitOps. Admins can declare their cluster configuration and applications in Git. Development teams can then use pull requests and other tools they're familiar with (existing Azure Pipelines, Git, Kubernetes manifests, Helm charts) to easily deploy applications into Azure Arc-enabled Kubernetes clusters and make updates in production. Billing is charged monthly and is based on the number of vCPUs/hour in your cluster. Clusters incur a single charge for configuration management, no matter how many repositories are connected.

  [!Note] Clusters can function without a constant connection to Azure. When disconnected, each cluster’s charge is determined based on the last known number of vCPUs that were registered with Azure Arc. The vCPU count is updated every 5 minutes while your cluster is connected to Azure. Each cluster's first 6 vCPUs are included at no cost.

  If your cluster will be disconnected from Azure and you don’t want to be charged for Kubernetes configurations, you can delete the configurations.

• Azure Policy for Kubernetes: Azure Policy for Kubernetes extends Gatekeeper v3, an admission controller webhook for Open Policy Agent (OPA), to apply at-scale enforcements and safeguards on your clusters in a centralized, consistent manner. Azure Policy makes it possible to manage and report on the compliance state of your Kubernetes clusters from one place. There's currently no cost for Azure Policy for Kubernetes while in public preview.

• Microsoft Sentinel: Microsoft Sentinel provides intelligent security analytics across your enterprise. The data for its analysis is stored in an Azure Monitor Log Analytics workspace. Microsoft Sentinel is billed based on the volume of data ingested for analysis in Microsoft Sentinel and stored in the Azure Monitor Log Analytics workspace for your Azure Arc-enabled Kubernetes clusters.

• Azure Key Vault: The Azure Key Vault Provider for Secrets Store CSI Driver allows for the integration of an Azure Key Vault as a store of secrets with a Kubernetes cluster via a CSI volume. Azure Key Vault is billed by the operations performed on certificates, keys, and secrets.

The following sections contain design recommendations for Azure Arc-enabled Kubernetes cost governance.

• Review the recommendations in the resource organization and governance disciplines critical design area to implement a governance strategy, organize your resources for better cost control and visibility, and avoid unnecessary costs by using the least privileged access model for onboarding and management.

• Review the Management and monitoring critical design area to plan your monitoring strategy and decide on your requirements for monitoring Azure Arc-enabled Kubernetes clusters to optimize costs.

• Review Azure Monitor for Containers pricing.

• Use the Azure Pricing Calculator to get an estimate of Azure Arc-enabled Kubernetes monitoring costs for Azure Log Analytics ingestion, alerts, and notifications.

  :::image type="content" source="./media/arc-enabled-kubernetes-cost-pricing-calculator.png" alt-text="A screenshot showing Azure Pricing Calculator." lightbox="./media/arc-enabled-kubernetes-cost-pricing-calculator.png" border="true":::

  :::image type="content" source="./media/arc-enabled-kubernetes-cost-pricing-calculator-az-monitor.png" alt-text="A screenshot showing Azure Pricing Calculator Azure Monitor." lightbox="./media/arc-enabled-kubernetes-cost-pricing-calculator-az-monitor.png" border="true":::

• Use Microsoft Cost Management to view the costs of Azure Monitor for Containers.

  :::image type="content" source="./media/arc-enabled-kubernetes-cost-billing.png" alt-text="A screenshot showing Microsoft Cost Management." lightbox="./media/arc-enabled-kubernetes-cost-billing.png" border="true":::

• Use the Log Analytics workspace insights solution to gain insights on monitored Azure Kubernetes clusters, collected logs, and their integration rate so you can avoid unnecessary ingestion costs.

  :::image type="content" source="./media/arc-enabled-kubernetes-cost-log-analytics-insights.png" alt-text="A screenshot showing Log Analytics insights." lightbox="./media/arc-enabled-kubernetes-cost-log-analytics-insights.png" border="true":::

• Use built-in Azure Monitor workbooks to understand your clusters' billable monitoring data.

  :::image type="content" source="./media/arc-enabled-kubernetes-cost-monitor-workbooks.png" alt-text="A screenshot showing Azure Monitor workbook showing namespace billing." lightbox="./media/arc-enabled-kubernetes-cost-monitor-workbooks.png" border="true":::

• Review tips for reducing Log Analytics ingestion data volume to help you properly configure data ingestion.

• Consider how long you should retain data in Log Analytics. Data ingested into the Log Analytics workspace can be retained for no extra charge up through the first 31 days. Consider general needs when configuring the Log Analytics workspace level default retention and specific needs when configuring data retention by data type, which can be as low as four days. For example, while performance data might only need to be retained for a short time, security logs often need to be retained longer.

• Consider using Log Analytics workspace data export to retain data for longer than 730 days.

• Consider using Commitment Tier pricing based on your data ingestion volume.

• Review the Security, governance, and compliance critical design area to understand how to use Microsoft Defender for Cloud to protect and secure your Azure Arc-enabled Kubernetes clusters.
• Review Microsoft Defender for Containers pricing information.
• Consider deploying the Microsoft Defender for Containers cost estimation workbook to understand the cost estimates for using Microsoft Defender for Containers to protect your Azure Arc-enabled Kubernetes cluster(s).

• Review Kubernetes GitOps configuration pricing.

• Review the CI/CD workflow critical design area to find best practices and recommendations for managing and monitoring Kubernetes GitOps configuration on your Azure Arc-enabled Kubernetes clusters.

• Use Azure Policy for Kubernetes to enforce and ensure consistent configuration across all your Azure Arc-enabled Kubernetes clusters.

• Use Azure Resource Graph queries to review the number of cores you have for Azure Arc-enabled Kubernetes clusters and estimate the cost of enabling Kubernetes GitOps configuration.

  Resources
  | extend AgentVersion=properties.agentVersion, KubernetesVersion=properties.kubernetesVersion, Distribution= properties.distribution,Infrastructure=properties.infrastructure, NodeCount=properties.totalNodeCount,TotalCoreCount=toint(properties.totalCoreCount)
  | project id, subscriptionId, location, type,AgentVersion ,KubernetesVersion ,Distribution,Infrastructure ,NodeCount , TotalCoreCount
  | where type =~ 'Microsoft.Kubernetes/connectedClusters'
  | order by TotalCoreCount

• Use Microsoft Cost Management to understand Kubernetes GitOps configuration costs.

  :::image type="content" source="./media/arc-enabled-kubernetes-configuration-cost.png" alt-text="A screenshot showing Kubernetes GitOps configuration cost in Azure portal." lightbox="./media/arc-enabled-kubernetes-configuration-cost.png" border="true":::

• Review Azure Policy for Kubernetes pricing.
• Review the Security, governance, and compliance critical design area to learn best practices and recommendations for implementing Azure Policy for Kubernetes. These best practices include:
  • Enforcing tagging for better cost visibility across clusters
  • Enforcing Kubernetes GitOps configuration
  • Controlling the enablement of Azure services.

• Review Microsoft Sentinel pricing.
• Use the Azure Pricing Calculator to estimate Microsoft Sentinel costs for your organization.

• Use Microsoft Sentinel Cost Management and Billing to understand Microsoft Sentinel analysis costs.

  :::image type="content" source="./media/arc-enabled-kubernetes-cost-sentinel.png" alt-text="A screenshot showing Microsoft Sentinel cost analysis." lightbox="./media/arc-enabled-kubernetes-cost-sentinel.png" border="true":::

• Review data retention costs for data ingested into the Log Analytics workspace Microsoft Sentinel uses.

• Filter the right level of logs and events for your Azure Arc-enabled Kubernetes clusters to be collected in the Log Analytics workspace.

• Use Log Analytics queries and the Workspace Usage Report workbook to understand your data ingestion trends.

• Create a cost management playbook to send notifications if your Microsoft Sentinel workspace exceeds your budget.

• Microsoft Sentinel integrates with other Azure services to provide enhanced capabilities. Review the pricing details for these services.

• Consider using Commitment Tier pricing based on your data ingestion volume.

• Consider separating non-security operational data into a different Azure Log Analytics workspace.

• Review Azure Key Vault pricing.

• Review recommendations for security and governance to understand how you can use Azure Key vault to manage secrets and certificates on your Azure Arc-enabled Kubernetes clusters.

• Use Azure Key Vault insights to monitor secrets operations.

  :::image type="content" source="./media/arc-enabled-kubernetes-cost-key-vault-insights.png" alt-text="A screenshot showing Azure Key Vault insights" lightbox="./media/arc-enabled-kubernetes-cost-key-vault-insights.png" border="true":::

For more information about your hybrid and multicloud cloud journey, see the following articles:

• Review the prerequisites for Azure Arc-enabled Kubernetes.
• Review the validated Kubernetes distributions for Azure Arc-enabled Kubernetes.
• Learn how to Manage hybrid and multicloud environments.
• Experience Azure Arc-enabled Kubernetes automated scenarios with Azure Arc Jumpstart.
• Learn more about Azure Arc via the Azure Arc learning path.
• Review the Cloud Adoption Framework best practices and recommendations to efficiently manage your cloud costs.
• See the Frequently Asked Questions - Azure Arc-enabled to find answers to most common questions.