or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

alb-controller.mdauth.mdcluster.mdfargate.mdindex.mdkubectl-provider.mdkubernetes-resources.mdnode-groups.mdservice-accounts.mdutilities.md
tile.json

kubectl-provider.mddocs/

Kubectl Provider

Custom resource provider for executing kubectl operations in EKS clusters, enabling direct Kubernetes resource management through AWS Lambda functions.

Capabilities

KubectlProvider Class

Lambda-based custom resource provider for kubectl operations.

/**
 * Custom resource provider for kubectl operations
 */
class KubectlProvider extends NestedStack implements IKubectlProvider {
  constructor(scope: Construct, id: string, props: KubectlProviderProps);
  
  /** Custom resource provider's service token */
  readonly serviceToken: string;
  /** IAM role ARN for kubectl operations */
  readonly roleArn: string;
  /** IAM execution role of the handler */
  readonly handlerRole: iam.IRole;
  
  /** Get or create kubectl provider for cluster */
  static getOrCreate(scope: Construct, cluster: ICluster): IKubectlProvider;
  /** Import existing kubectl provider */
  static fromKubectlProviderAttributes(scope: Construct, id: string, attrs: KubectlProviderAttributes): IKubectlProvider;
}

/**
 * Kubectl provider interface
 */
interface IKubectlProvider extends IConstruct {
  /** Custom resource provider's service token */
  readonly serviceToken: string;
  /** IAM role ARN for kubectl operations */
  readonly roleArn: string;
  /** IAM execution role of the handler */
  readonly handlerRole: iam.IRole;
}

/**
 * Kubectl provider properties
 */
interface KubectlProviderProps {
  /** EKS cluster to control */
  readonly cluster: ICluster;
}

/**
 * Kubectl provider import attributes
 */
interface KubectlProviderAttributes {
  /** Kubectl provider lambda ARN */
  readonly functionArn: string;
  /** IAM role ARN for kubectl operations */
  readonly kubectlRoleArn: string;
  /** IAM execution role of the handler */
  readonly handlerRole: iam.IRole;
}

Usage Examples:

import * as eks from "@aws-cdk/aws-eks";

// Get or create kubectl provider (automatically managed)
const provider = eks.KubectlProvider.getOrCreate(this, cluster);

// Import existing kubectl provider
const importedProvider = eks.KubectlProvider.fromKubectlProviderAttributes(this, "ImportedProvider", {
  functionArn: "arn:aws:lambda:us-west-2:123456789012:function:MyKubectlHandler",
  kubectlRoleArn: "arn:aws:iam::123456789012:role/MyKubectlRole",
  handlerRole: existingRole,
});

// Use provider for custom resources
const customResource = new CustomResource(this, "CustomK8sResource", {
  serviceToken: provider.serviceToken,
  properties: {
    ClusterName: cluster.clusterName,
    RoleArn: provider.roleArn,
    Manifest: JSON.stringify({
      apiVersion: "v1",
      kind: "ConfigMap",
      metadata: {
        name: "my-config",
        namespace: "default",
      },
      data: {
        "config.yaml": "key: value",
      },
    }),
  },
});

Provider Architecture

The kubectl provider creates a Lambda-based custom resource with the following components:

Lambda Function Configuration:

  • Runtime: Python 3.7
  • Timeout: 15 minutes
  • Memory: Configurable via cluster.kubectlMemory (default: 1024 MB)
  • Layers: AWS CLI and kubectl layers
  • Environment: Configured via cluster.kubectlEnvironment

IAM Permissions:

  • eks:DescribeCluster on the target cluster
  • sts:AssumeRole on the cluster's kubectl role
  • AmazonEC2ContainerRegistryReadOnly for OCI Helm charts
  • VPC execution role if deployed in private subnets

Network Configuration:

  • Deployed in VPC if cluster.kubectlPrivateSubnets is specified
  • Uses cluster.kubectlSecurityGroup for network access
  • Supports private endpoint access

Static Methods

/**
 * Get existing provider or create new one for cluster
 * @param scope - Construct scope
 * @param cluster - EKS cluster
 * @returns Kubectl provider instance
 */
static getOrCreate(scope: Construct, cluster: ICluster): IKubectlProvider;

/**
 * Import existing kubectl provider
 * @param scope - Construct scope
 * @param id - Resource identifier
 * @param attrs - Provider attributes
 * @returns Imported kubectl provider
 */
static fromKubectlProviderAttributes(
  scope: Construct,
  id: string,
  attrs: KubectlProviderAttributes
): IKubectlProvider;

Integration with Kubernetes Resources

The kubectl provider is automatically used by the following EKS constructs:

  • KubernetesManifest - Apply raw Kubernetes manifests
  • HelmChart - Install Helm charts
  • KubernetesPatch - Apply patches to resources
  • KubernetesObjectValue - Query resource values
  • AwsAuth - Manage aws-auth ConfigMap
// These constructs automatically use the kubectl provider
cluster.addManifest("MyManifest", {
  apiVersion: "v1",
  kind: "Pod",
  metadata: { name: "my-pod" },
  spec: {
    containers: [{ name: "app", image: "nginx" }],
  },
});

cluster.addHelmChart("MyChart", {
  chart: "stable/nginx-ingress",
  repository: "https://kubernetes-charts.storage.googleapis.com/",
});

Custom Resource Operations

The kubectl provider supports the following operations:

Create/Update Operations:

  • Apply Kubernetes manifests
  • Install/upgrade Helm charts
  • Patch existing resources
  • Create/update aws-auth ConfigMap entries

Delete Operations:

  • Delete Kubernetes resources
  • Uninstall Helm charts
  • Remove aws-auth ConfigMap entries

Query Operations:

  • Get resource values using JSONPath
  • Query resource status
  • Validate resource existence

Error Handling

The provider includes comprehensive error handling:

// Provider will retry operations with exponential backoff
// Errors are surfaced as CloudFormation custom resource failures
// Logs are available in CloudWatch for troubleshooting

// Example error scenarios:
// - Cluster unreachable
// - Insufficient RBAC permissions
// - Invalid Kubernetes manifests
// - Helm chart installation failures

Best Practices

  1. Resource Lifecycle: Let CDK manage the provider lifecycle automatically
  2. Memory Configuration: Adjust kubectl memory for large manifests or Helm charts
  3. Network Security: Use private subnets and security groups for sensitive operations
  4. Error Monitoring: Monitor CloudWatch logs for kubectl operation failures
  5. RBAC Permissions: Ensure kubectl role has necessary permissions for operations
// Best practice: Configure cluster for private kubectl access
const cluster = new eks.Cluster(this, "PrivateCluster", {
  version: eks.KubernetesVersion.V1_21,
  kubectlMemory: Size.mebibytes(2048),
  kubectlPrivateSubnets: vpc.privateSubnets,
  kubectlSecurityGroup: kubectlSecurityGroup,
  kubectlEnvironment: {
    "CLUSTER_NAME": "my-private-cluster",
  },
});

Troubleshooting

Common Issues:

  1. Provider timeout: Increase memory or timeout for large operations
  2. Network connectivity: Verify security group and subnet configuration
  3. Permission errors: Check kubectl role RBAC permissions
  4. Resource conflicts: Handle existing resources before applying manifests

Debugging:

// Enable detailed logging
const cluster = new eks.Cluster(this, "Cluster", {
  kubectlEnvironment: {
    "AWS_LAMBDA_LOG_LEVEL": "DEBUG",
  },
});

// Check CloudWatch logs at:
// /aws/lambda/[stack-name]-[cluster-id]-KubectlProvider-Handler