Deploying agents

Deploy Flyte agents to a Kubernetes cluster, the Flyte sandbox, and configure FlytePropeller to route task types to the agent service.

After you have developed and tested your agent locally, you need to deploy it to a Kubernetes cluster. This page covers deploying agents to the Flyte sandbox for integration testing and to a production cluster.

Deploying to the Flyte sandbox

The Flyte sandbox (flytectl demo start) is the easiest way to test an agent end-to-end before moving to a production cluster.

1. Start the Flyte sandbox

   flytectl demo start
   
2. Build your agent image

   Create a Dockerfile for your agent. The following example is for the Databricks agent:

   FROM python:3.9-slim-bookworm
   
   RUN apt-get update && apt-get install build-essential git -y
   RUN pip install prometheus-client grpcio-health-checking
   RUN pip install --no-cache-dir -U flytekit \\
       git+https://github.com/flyteorg/flytekit.git@<gitsha>#subdirectory=plugins/flytekit-spark \\
       && apt-get clean autoclean \\
       && apt-get autoremove --yes \\
       && rm -rf /var/lib/{apt,dpkg,cache,log}/
   
   CMD pyflyte serve agent --port 8000
   

   Build and push the image to the sandbox registry:

   docker buildx build -t localhost:30000/flyteagent:example -f Dockerfile.agent . --load
   docker push localhost:30000/flyteagent:example
   
3. Deploy the agent image to the cluster

   Edit the flyteagent deployment to use your new image:

   kubectl edit deployment flyteagent -n flyte
   

   Find the image key and update its value:

   image: localhost:30000/flyteagent:example
   
4. Configure secrets

   Store agent credentials as Kubernetes secrets. The agent deployment mounts the flyteagent secret in the flyte namespace. Use the following command to add a secret value (the Databricks token is used here as an example):

   SECRET_VALUE=$(echo -n "<DATABRICKS_TOKEN>" | base64) && \\
   kubectl patch secret flyteagent -n flyte \\
     --patch "{\"data\":{\"flyte_databricks_access_token\":\"$SECRET_VALUE\"}}"
   

   Secret names must contain only lowercase English letters. Secret values must be Base64-encoded.

5. Restart the agent deployment

   Apply the new image and secrets by restarting the deployment:

   kubectl rollout restart deployment flyteagent -n flyte
   
6. Test your agent remotely

   Run a workflow that uses the agent:

   pyflyte run --remote agent_workflow.py agent_task
   

   You must build an image that includes the plugin package for the task and specify it with the --image flag or via an ImageSpec definition in your workflow file.

Managing secrets

All agent secrets are stored in the flyteagent Kubernetes secret in the flyte namespace. To add a secret for any agent:

SECRET_VALUE=$(<YOUR_SECRET_VALUE> | base64) && \
kubectl patch secret flyteagent -n flyte \
  --patch "{\"data\":{\"your_agent_secret_name\":\"$SECRET_VALUE\"}}"

Always Base64-encode secret values before patching. Storing plaintext secrets in the Kubernetes secret object will cause authentication failures.

Deploying to a production cluster

For production deployments, update the flyteagent deployment’s image directly:

kubectl set image deployment/flyteagent flyteagent=ghcr.io/flyteorg/flyteagent:latest

Ensure your FlytePropeller image version is >= 1.13.0. Propeller 1.13.0+ automatically fetches the list of supported task types from the agent deployment, removing the need to manually specify task types in the configmap.

Configuring FlytePropeller to use the agent service

FlytePropeller needs to know the endpoint of the agent service. Configure this in the propeller configmap:

plugins:
  agent-service:
    defaultAgent:
      endpoint: "k8s://flyteagent.flyte:8000"
      insecure: true
      timeouts:
        # CreateTask, GetTask, DeleteTask are for async agents
        CreateTask: 5s
        GetTask: 5s
        DeleteTask: 5s
        # ExecuteTaskSync is for sync agents
        ExecuteTaskSync: 10s
      defaultTimeout: 10s

Routing task types to custom agent services

If you have multiple agent services (for example, a dedicated service for a custom integration), configure routing rules:

plugins:
  agent-service:
    defaultAgent:
      endpoint: "k8s://flyteagent.flyte:8000"
      insecure: true
      defaultTimeout: 10s
    agents:
      custom_agent:
        endpoint: "dns:///custom-flyteagent.flyte.svc.cluster.local:8000"
        insecure: false
        defaultServiceConfig: '{"loadBalancingConfig": [{"round_robin":{}}]}'
        timeouts:
          GetTask: 5s
        defaultTimeout: 10s
    agentForTaskTypes:
      # Requests for custom_task go to custom_agent; all other types go to defaultAgent
      - custom_task: custom_agent

Enabling agents in a managed deployment

If you are using a managed Flyte deployment, you cannot modify the agent configuration directly. Contact your deployment administrator to:

1. Enable the agent plugin package in the agent service image.
2. Add any required secrets to the flyteagent Kubernetes secret.
3. Restart the agent deployment.

For per-agent configuration guides, see the Agent setup documentation.

Verifying the deployment

After deploying, verify the agent is running and reachable:

# Check that the agent pod is running
kubectl get pods -n flyte -l app=flyteagent

# Tail agent logs for debugging
kubectl logs -n flyte -l app=flyteagent -f

# Check the rollout status
kubectl rollout status deployment/flyteagent -n flyte

Enable DEBUG logging in the agent service to trace gRPC calls from FlytePropeller. Set the FLYTE_INTERNAL_LOG_LEVEL=DEBUG environment variable in the agent deployment spec.