> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-feature-comparison-update.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# EKS

> This enterprise-focused document provides comprehensive instructions for deploying the Portkey software in a hybrid mode on Amazon EKS clusters, designed to meet the needs of large-scale, mission-critical applications. It includes specific recommendations for component sizing, high availability, and integration with monitoring systems.

## Components and Sizing Recommendations

| Component                            | Options                                                    | Sizing Recommendations                                                                                                                                              |
| ------------------------------------ | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| AI Gateway                           | Deploy in your EKS cluster using Helm charts.              | Use Amazon EKS t4g.medium worker nodes, each providing at least 2 vCPUs and 4 GiB of memory. For high availability, deploy them across multiple Availability Zones. |
| Logs Store (optional)                | Amazon S3 or S3-compatible Storage                         | Each log document is \~10kb in size (uncompressed)                                                                                                                  |
| Cache (Prompts, Configs & Providers) | Built-in Redis, Amazon ElastiCache for Redis OSS or Valkey | Deployed within the same VPC as the Portkey Gateway.                                                                                                                |

## Prerequisites

Ensure that following tools and resources are installed and available:

* A running [Amazon EKS cluster](https://docs.aws.amazon.com/eks/latest/userguide/create-cluster-auto.html) with at least 2 worker nodes. ( **Best Practice:** Use 2 nodes, with 1 node in each Availability Zone, to ensure high availability.)
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
* [Kubectl](https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html)
* [Helm (v3 or above)](https://helm.sh/docs/intro/install/)
* [eksctl](https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html)

## Create a Portkey Account

* Go to the [Portkey](https://app.portkey.ai) website.
* Sign up for a Portkey account.
* Once logged in, locate and save your `Organisation ID` for future reference. You can find it in the browser URL:
  `https://app.portkey.ai/organisation/<organisation_id>/`
* Contact the Portkey AI team and provide your Organisation ID and the email address used during signup.
* The Portkey team will share the following information with you:
  * Docker credentials for the Gateway images (username and password).
  * License: Client Auth Key.

## Setup Project Environment

```sh theme={null}
cluster_name=<EKS_CLUSTER_NAME>               # Specify the name of the EKS cluster where the gateway will be deployed.
namespace=<NAMESPACE>                         # Specify the namespace where the gateway should be deployed (for example, portkeyai).
service_account_name=<SERVICE_ACCOUNT_NAME>   # Provide a name for the Service Account to be associated with Gateway Pod (for example, gateway-sa)

mkdir portkey-gateway
cd portkey-gateway
touch values.yaml
```

### Image Credentials Configuration

```yaml theme={null}
# Update the values.yaml file
imageCredentials:
  - name: portkey-enterprise-registry-credentials
    create: true
    registry: https://index.docker.io/v1/
    username: <PROVIDED BY PORTKEY>
    password: <PROVIDED BY PORTKEY>

  gatewayImage:
    repository: "docker.io/portkeyai/gateway_enterprise"
    pullPolicy: Always
    tag: "latest"
  dataserviceImage:
    repository: "docker.io/portkeyai/data-service"
    pullPolicy: Always
    tag: "latest"
  redisImage:
    repository: "docker.io/redis"
    pullPolicy: IfNotPresent
    tag: "7.2-alpine"
environment:
  create: true
  secret: true
  data:
    ANALYTICS_STORE: control_plane
    SERVICE_NAME: <SERVICE_NAME>                      # Specify a name for the service
    PORTKEY_CLIENT_AUTH: <PROVIDED BY PORTKEY>
    ORGANISATIONS_TO_SYNC: <ORGANISATION_ID>           # This is obtained after signing up for a Portkey account.  
```

## Configure Components

Based on the choice of components and their configuration update the `values.yaml`.

### MCP Gateway (Optional)

By default, only the AI Gateway is enabled in the deployment. To enable the MCP Gateway, add the following configuration to `values.yaml`:

```yaml theme={null}
environment:
  data:
    SERVER_MODE: "mcp/all"
    MCP_PORT: "8788"
    MCP_GATEWAY_BASE_URL: "<This must be set to MCP LoadBalancer URL or Domain pointing to MCP Service>"
```

**Note:**

* `MCP_GATEWAY_BASE_URL` must include the protocol prefix — either `http://` or `https://`.
* This value is not required for the initial deployment. After the first deployment, once the MCP Load Balancer is provisioned and a hostname is mapped to the MCP Service, set this value and redeploy.

**Server Modes**

1. `""` (empty or not provided): Deploys only the AI Gateway. This is the default configuration.
2. `"mcp"`: Deploys only the MCP Gateway.
3. `"all"`: Deploys both the AI Gateway and MCP Gateway.

### Cache Store

The Portkey Gateway deployment includes a Redis instance pre-installed by default. You can either use this built-in Redis or connect to an external cache like `Amazon ElastiCache for Redis OSS` or `Valkey`.

#### Built-in Redis

No additional permissions or network configurations are required.

```yaml theme={null}
## To use the built-in Redis, add the following configuration to the values.yaml file.
environment:
  data:
    CACHE_STORE: redis
    REDIS_URL: "redis://redis:6379"
    REDIS_TLS_ENABLED: "false"
```

#### Amazon ElastiCache

To enable the gateway to work with an ElastiCache cache, ensure that inbound rule is configured in ElastiCache's Security Group allowing access from EKS cluster on required port.

```yaml theme={null}
## To use Amazon ElastiCache for Redis OSS or Valkey, add the following configuration in the values.yaml file.
environment:
  data:
    CACHE_STORE: aws-elastic-cache
    REDIS_URL: "redis://<ElastiCache_Endpoint>:<Port>" 
    REDIS_TLS_ENABLED: "true"                             ## "true"/"false"
    REDIS_MODE: cluster                                   ## Add this parameter only if cluster mode is enabled on Amazon ElastiCache
    # REDIS_PASSWORD: <Auth Token>                        ## Provide Auth Token if enabled on Amazon ElastiCache

```

**Note:** If cluster mode is enabled in ElastiCache then use **Configuration Endpoint** otherwise use **Primary Endpoint**.
For more information on ElastiCache endpoints, refer to the [AWS resources](https://docs.aws.amazon.com/AmazonElastiCache/latest/dg/Endpoints.html).

### Log Store

#### Amazon S3

1. Create an Amazon S3 bucket for storing LLM access logs.

2. [Set up](#setting-up-iam-permission) access to the log store. The Gateway supports the following methods for connecting to S3 bucket for log storage:

   * IAM Roles for Service Accounts (IRSA)
   * EKS Pod Identity

   Depending on the chosen S3 access method, update `values.yaml` with the following configuration.

   <Tabs>
     <Tab title="IRSA">
       ```yaml theme={null}
       ## To enable IRSA update values.yaml with the following details:-
       serviceAccount:
         create: true
         automount: true
         name: <SERVICE_ACCOUNT_NAME>             # Provide the name of service account. Must be same as the name you provided while creating IAM Role in last step.
         annotations:
           eks.amazonaws.com/role-arn: <ROLE_ARN> # Provide the IAM role ARN obtained in previous step.

       environment:
         data:
           LOG_STORE: s3_assume
           LOG_STORE_REGION: "<AWS_BUCKET_REGION>"                     # Specify the AWS region where the S3 log bucket resides (e.g., us-east-1).
           LOG_STORE_GENERATIONS_BUCKET: "<AWS_BUCKET_NAME>"           # Specify the name of S3 log bucket.
       ```
     </Tab>

     <Tab title="EKS Pod Identity">
       ```yaml theme={null}
       ## To enable EKS Pod Identity update values.yaml with following details:-
       serviceAccount:
         create: true
         automount: true
         name: <SERVICE_ACCOUNT_NAME>             # Provide the name of service account. Must be same as the name you provided while creating IAM Role in last step.

       environment:
         data:
           LOG_STORE: s3_assume
           LOG_STORE_REGION: "<AWS_BUCKET_REGION>"                     # Specify the AWS region where the S3 log bucket resides (e.g., us-east-1).
           LOG_STORE_GENERATIONS_BUCKET: "<AWS_BUCKET_NAME>"           # Specify the name of S3 log bucket.
       ```
     </Tab>
   </Tabs>

3. (Optional) Configure log path format using `LOG_STORE_FILE_PATH_FORMAT`. See [Log Object Path Format](/product/enterprise-offering/components#log-object-path-format) for details.

### Data Service (Optional)

The Data Service is a component of the Portkey deployment responsible for batch processing, fine-tuning, and log exports.

To enable Data Service, add the following configuration to the `values.yaml` file.

```yaml theme={null}
dataservice:
  name: "dataservice"
  enabled: true
  env:
    DEBUG_ENABLED: false
    SERVICE_NAME: "portkeyenterprise-dataservice"
  serviceAccount:
    create: false
    name: <SERVICE_ACCOUNT_NAME>                                  # Provide the name of service account. Must be same as the name you provided while creating IAM Role in last step.


```

## Network Configuration

### Set Up External Access

To make the Gateway service accessible externally, you can set up either of the following:

* **AWS Application Load Balancer** with Kubernetes `Ingress`
* **AWS Network Load Balancer** with Kubernetes `Service`

**Prerequisites**

* VPC and subnet [tagging requirements](https://docs.aws.amazon.com/eks/latest/userguide/network-reqs.html)
* Installed and running AWS Load Balancer Controller. For Load Balancer Controller installation details, refer to the AWS [documentation](https://docs.aws.amazon.com/eks/latest/userguide/lbc-helm.html).

#### AWS Application Load Balancer

To create Application Load Balancer Ingress update the `values.yaml` file with following configuration:

```yaml theme={null}
service:
  type: ClusterIP
  port: 8787

ingress:
  enabled: true
  # hostname: "<AI Gateway Hostname>"
  # hostBased: false
  # mcpHostname: "<MCP Gateway Hostname>"
  ingressClassName: "alb"
  annotations: 
    alb.ingress.kubernetes.io/load-balancer-name : portkey-gateway
    alb.ingress.kubernetes.io/scheme: internet-facing                                                 # Set to 'internal' for internal ALB, set to 'internet-facing' for creating an ALB accessible from internet.
    alb.ingress.kubernetes.io/target-type: ip 
    # alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
    alb.ingress.kubernetes.io/healthcheck-path: /v1/health
    alb.ingress.kubernetes.io/inbound-cidrs: 0.0.0.0/0                                                # Allowed inbound CIDR ranges
    alb.ingress.kubernetes.io/manage-backend-security-group-rules: "true"
```

**Note:** If `SERVER_MODE` is set to `all` (i.e., both AI Gateway and MCP Gateway are enabled), you must enable host-based routing by setting `hostBased` to `true` and provide the hostname on which the AI Gateway and MCP Gateway will be accessible.

Load Balancer Controller provides additional annotations (like TLS, custom health checks etc ) for managing ALB. For a comprehensive list of available annotations, refer to the [AWS Load Balancer Controller documentation](https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/ingress/annotations/).

#### AWS Network Load Balancer

To create Network Balancer update the `values.yaml` with following configuration:

```yaml theme={null}
service:
  type: LoadBalancer
  port: 80                                                                                          # NLB listener port                                                 
  containerPort: 8787                                             
  annotations: 
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"                   
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"                                   # Set to 'true' to create an internal NLB, set to 'false' to create an internet-facing NLB.
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"     
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: "/v1/health"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: "http" 
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "8787"        
    service.beta.kubernetes.io/aws-load-balancer-manage-backend-security-group-rules: "true"         
```

**Note:** `service.containerPort` must be same as `environment.data.PORT`.

Load Balancer Controller provides additional annotations (like TLS, custom health checks etc ) for managing NLB. For a comprehensive list of available annotations, refer to the [AWS Load Balancer Controller documentation](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.7/guide/service/annotations/).

## Deploying Portkey Gateway

```sh theme={null}
# Add the Portkey AI Gateway helm repository
helm repo add portkey-ai https://portkey-ai.github.io/helm
helm repo update

# Install the chart
helm upgrade --install portkey-ai portkey-ai/gateway -f ./values.yaml -n $namespace --create-namespace
```

## Verify the deployment

To confirm that the deployment was successful, follow these steps:

* Verify that all pods are running correctly.

```sh theme={null}
# 
kubectl get pods -n $namespace
# You should see all pods with a 'STATUS' of 'Running'.
```

**Note:** If pods are in a Pending, CrashLoopBackOff, or other error state, inspect the [pod logs](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_logs/) and [events](https://kubernetes.io/docs/reference/kubectl/generated/kubectl_events/) to diagnose potential issues.

* Test Gateway by sending a cURL request.

  1. Port-forward the Gateway pod

  ```sh theme={null}
    kubectl port-forward  <POD_NAME> -n $namespace 9000:8787       # Replace <POD_NAME> with your Gateway pod's actual name.
  ```

  2. Once port forwarding is active, open a new terminal window or tab and send a test request by running:

  ```sh theme={null}
  # Specify LLM provider and Portkey API keys
  OPENAI_API_KEY=<OPENAI_API_KEY>                           # Replace <OPENAI_API_KEY> with an actual API key
  PORTKEY_API_KEY=<PORTKEY_API_KEY>                         # Replace <PORTKEY_API_KEY> with Portkey API key which can be created from Portkey website(https://app.portkey.ai/api-keys).

  # Configure and send the curl request
  curl 'http://localhost:9000/v1/chat/completions'`\
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY"  \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY"  \
  -d '{ 
      "model": "gpt-4o-mini", 
      "messages": [{"role": "user","content": "What is a fractal?"}]  
  }'
  ```

  3. Test gateway service integration with Load Balancer.

  ```sh theme={null}
  # Replace <LOAD_BALANCER_IP> and <LISTENER_PORT> with the Load Balancer's IP/DNS and listener port respectively. 

  curl 'http://<LB_IP>:<LISTENER_PORT>/v1/chat/completions' \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY"  \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY"  \
  -d '{
      "model": "gpt-4o-mini",
      "messages": [{"role": "user","content": "What is a fractal?"}]
  }'
  ```

## Integrating Gateway with Control Plane

**Outbound Connectivity (Data Plane to Control Plane)**

Portkey supports the following methods for integrating the Data Plane with the Control Plane for outbound connectivity:

* AWS PrivateLink
* Over the Internet

**Ensure Outbound Network Access**

By default, Kubernetes allows full outbound access, but if your cluster has NetworkPolicies that restrict egress, configure them to allow outbound traffic.

Example NetworkPolicy for Outbound Access:

```yaml theme={null}
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-all-egress
  namespace: portkeyai
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
```

This allows the gateway to access LLMs hosted both within your VPC and externally. This also enables connection for the sync service to the Portkey Control Plane.

#### AWS PrivateLink

Establishes a secure, private connection between the Control Plane and Data Plane within the AWS network.

**Steps to establish AWS PrivateLink connectivity:**

1. Contact Portkey and provide AWS account ARN so it can be whitelisted in Portkey's Control Plane.
2. Once you get confirmation from Portkey that your AWS account is whitelisted, go to the [VPC Console](https://console.aws.amazon.com/vpc/).
3. Select the AWS Region where the Portkey Gateway is deployed.
4. Navigate to the **Endpoints** section in the VPC console.
5. Click on **Create endpoint** and enter the required details.
6. Select the `PrivateLink Ready partner services` category and, under **Service settings**, provide the following details.
   * For **Service name**, enter `com.amazonaws.vpce.us-east-1.vpce-svc-0c2c1c323d9f56d95`
   * (Optional) If the Gateway is deployed in a region other than `us-east-1`, select `Enable Cross Region endpoint`, choose the `us-east-1` region, and click the **Verify service** button.
7. Under **Network settings**
   * Select the VPC and subnets (at least two in different AZs for high availability) where the endpoint should be created. Ideally, this should be the same VPC where the Gateway is deployed.
   * Select the security group to associate with the endpoint. The security group must allow inbound connections on port 443 from the Gateway.
8. After all details are filled in, click on **Create endpoint**.
9. Wait for the Status to change to `Available`.
10. Once the status changes to `Available`, click on **Actions** > **Modify private DNS name** > Select **Enable for this endpoint**.
11. Update the `values.yaml` file with following config.
    ```yaml theme={null}
    environment:
      create: true
      secret: true
      data:
          ALBUS_BASEPATH: "https://aws-cp.portkey.ai/albus"
          CONTROL_PLANE_BASEPATH: "https://aws-cp.portkey.ai/api/v1"
          SOURCE_SYNC_API_BASEPATH: "https://aws-cp.portkey.ai/api/v1/sync"
          CONFIG_READER_PATH: "https://aws-cp.portkey.ai/api/model-configs" 
    ```
12. Re-deploy the gateway.
    ```sh theme={null}
    helm upgrade --install portkey-ai portkey-ai/gateway -f ./values.yaml  -n portkeyai  --create-namespace
    ```

#### Over the Internet

Ensure Gateway has access to following endpoints over the internet.

* `https://api.portkey.ai`
* `https://albus.portkey.ai`

### Inbound Connectivity (Control Plane to Data Plane)

* AWS PrivateLink
* IP Whitelisting

#### AWS PrivateLink

Establishes a secure, private connection between the Control Plane and Data Plane within the AWS network.

**Steps to establish AWS PrivateLink connectivity:**

To use AWS PrivateLink, you must create an AWS Network Load Balancer (NLB)—either internal or internet-facing—to expose the Gateway outside the EKS cluster.
For detailed instructions on creating and integrating an NLB, please refer to the [Networking Configuration](#network-configuration)

**Create Endpoint Service**

* Navigate to the [AWS VPC Console](https://ap-southeast-1.console.aws.amazon.com/vpcconsole/home#CreateVpcEndpointServiceConfiguration).
* In the top-right corner of the AWS Console, select the region where the Portkey Gateway is deployed.
* Provide the following details -
  * Name of endpoint service
  * Select Network Load Balancer to associate with Endpoint.
  * Choose region in which endpoint service will be available.
  * Select whether acceptance is required or not for requested connections.
  * Choose whether to enable private DNS name - If enabled provide the Private DNS Name.
  * Select **IPv4** under Supported IP address types.
* Click **Create**.

**(Optional) Verify ownership of Private DNS name**

This step needs to be performed if you are using Private DNS Name.

Open created Endpoint Service > click on **Actions** > select Verify domain ownership for private DNS name > Create the recommended record in your DNS server > Click Verify.

**Authorize Portkey's Control Plane to initiate connection requests**

* Open to Endpoint Service > click on **Actions** > select **Allow principals**, and enter the Control Plane's ARN(`arn:aws:iam::299329113195:root`).
  Reach out to portkey team and share the following details -
  * **Service name**
  * **DNS names**
  * **Private DNS name**
  * **Region** selected while creating Endpoint Service.
  * Port number on which Load Balancer is listening for connections.
* Wait for the Portkey team to initiate a connection request from the control plane's AWS account to your Gateway AWS account. Navigate to the **Endpoint connections** section and once the request appears, approve it.

#### IP Whitelisting

Allows control plane to access the Data Plane over the internet by restricting inbound traffic to specific IP address of Control Plane. This method requires the Data Plane to have a publicly accessible endpoint.
To whitelist, add an inbound rule to the Load Balancer's security group allowing connections from the Portkey Control Plane's IPs (`54.81.226.149`, `34.200.113.35`, `44.221.117.129`) on NLB listener port.

To integrate the Control Plane with the Data Plane, contact the Portkey team and provide the **Public Endpoint** of the Data Plane.

## Verifying Gateway Integration with the Control Plane

* Send a test request to Gateway using `curl`.
* Go to [Portkey website](https://app.portkey.ai/) -> **Logs**.
* Verify that the test request appears in the logs and that you can view its full details by selecting the log entry.

## Uninstalling Portkey Gateway

```sh theme={null}
helm uninstall portkey-ai --namespace $namespace
```

## Setting up IAM Permission

To enable the Portkey Gateway to access Amazon S3 for log storage and, optionally, Amazon Bedrock for model invocation, specific permissions are required.

Follow the steps below to configure permissions based on your chosen access method.

<Tabs>
  <Tab title="IRSA">
    1. Create an IAM trust policy to provide Gateway access to IAM Role.

    ```sh theme={null}
    bucket_name=<S3_BUCKET_NAME>                  # Specify the name of S3 bucket which will store logs. Bucket must already be created.
    role_name=<IAM_ROLE_NAME>                     # Provide a name for the role to be associated with Service Account.

    # Retrieve AWS Account ID
    aws_account_id=$(aws sts get-caller-identity --query Account --output text)
        
    # Retrieve EKS cluster’s OIDC issuer.
    oidc_issuer=$(aws eks describe-cluster --name $cluster_name --query "cluster.identity.oidc.issuer" --output text | sed -e "s~https://~~")

    # Check if an IAM OIDC provider is already created for EKS cluster in your account.
    aws iam list-open-id-connect-providers | grep $oidc_issuer

    # (Optional) If no output is returned, then create an IAM OIDC provider for your EKS cluster.
    eksctl utils associate-iam-oidc-provider --cluster $cluster_name --approve

    # Define a trust policy for IAM role
    cat >trust-relationship.json <<EOF
    {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "Federated": "arn:aws:iam::${aws_account_id}:oidc-provider/${oidc_issuer}"
        },
        "Action": "sts:AssumeRoleWithWebIdentity",
        "Condition": {
          "StringEquals": {
            "${oidc_issuer}:aud": "sts.amazonaws.com",
            "${oidc_issuer}:sub": "system:serviceaccount:${namespace}:${service_account_name}"
          }
        }
      }
    ]
    }
    EOF
    ```

    2. Create an IAM Role to associate with Gateway's service account.

    ```sh theme={null}
    # Create IAM role to associate with Gateway service account.                 
    aws iam create-role --role-name $role_name --assume-role-policy-document file://trust-relationship.json 

    # Fetch ARN of IAM role
    role_arn=$(aws iam get-role --role-name $role_name --query "Role.Arn" --output text)
    echo "$role_arn"
    ```

    **Note:** Record the IAM role ARN for future reference, as it will be required when configuring the Gateway’s service account in `values.yaml`.

    3. Attach an IAM policy to the role to grant access to the S3 log store and, optionally, Amazon Bedrock.

    **Amazon S3**

    ```sh theme={null}
    cat >s3-access-policy.json <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": ["s3:PutObject", "s3:GetObject"],
          "Resource": ["arn:aws:s3:::${bucket_name}/*"]
        }
      ]
    }
    EOF

    # Attach policy to the IAM role
    aws iam put-role-policy --role-name $role_name --policy-name s3-access-policy --policy-document file://s3-access-policy.json
    ```

    **(Optional) Amazon Bedrock**

    ```sh theme={null}
    cat >bedrock-access-policy.json <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
          "Resource": ["*"]
        }
      ]
    }
    EOF

    # Attach policy to the IAM role
    aws iam put-role-policy --role-name $role_name --policy-name bedrock-access-policy --policy-document file://bedrock-access-policy.json

    ```
  </Tab>

  <Tab title="EKS Pod Identity">
    1. (Optional) If the EKS Pod Identity Agent is not already installed on your cluster, install it before proceeding. For detailed, step-by-step instructions, refer to the following AWS [documentation](https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html).
    2. Create an IAM trust policy to provide Gateway access to IAM Role.

    ```sh theme={null}
    bucket_name=<S3_BUCKET_NAME>           # Specify the name of S3 bucket which will store logs. Bucket must already be created.
    role_name=<IAM_ROLE_NAME>              # Provide a name for the role to be associated with Service Account.
    cat >trust-relationship.json <<EOF
    {
        "Version": "2012-10-17",		 	 	 
        "Statement": [
            {
                "Sid": "AllowEksAuthToAssumeRoleForPodIdentity",
                "Effect": "Allow",
                "Principal": {
                    "Service": "pods.eks.amazonaws.com"
                },
                "Action": [
                    "sts:AssumeRole",
                    "sts:TagSession"
                ]
            }
        ]
    }
    EOF
    ```

    3. Create an IAM Role to associate with Gateway's service account.

    ```sh theme={null}
    aws iam create-role --role-name $role_name --assume-role-policy-document file://trust-relationship.json 
    ```

    4. Create a Pod Identity association that allows the Gateway’s service account to assume the specified IAM role using EKS Pod Identity.

    ```sh theme={null}
    # Retrieve AWS Account ID
    aws_account_id=$(aws sts get-caller-identity --query Account --output text)

    # Create Pod Identity Association
    aws eks create-pod-identity-association --cluster-name $cluster_name --role-arn "arn:aws:iam::${aws_account_id}:role/${role_name}" --namespace $namespace --service-account $service_account_name
    ```

    5. Attach an IAM policy to the role to grant access to the S3 log store and, optionally, Amazon Bedrock.

    **Amazon S3**

    ```sh theme={null}
    cat >s3-access-policy.json <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": ["s3:PutObject", "s3:GetObject"],
          "Resource": ["arn:aws:s3:::${bucket_name}/*"]
        }
      ]
    }
    EOF

    # Attach policy to the IAM role
    aws iam put-role-policy --role-name $role_name --policy-name s3-access-policy --policy-document file://s3-access-policy.json
    ```

    **(Optional) Amazon Bedrock**

    ```sh theme={null}
    cat >bedrock-access-policy.json <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
          "Resource": ["*"]
        }
      ]
    }
    EOF

    # Attach policy to the IAM role
    aws iam put-role-policy --role-name $role_name --policy-name bedrock-access-policy --policy-document file://bedrock-access-policy.json

    ```
  </Tab>
</Tabs>

## Examples

**Built-in Redis**

The following sample `values.yaml` below shows how to configure the built-in Redis cache and Amazon S3 log store using IRSA.

```yaml theme={null}
images:
  gatewayImage:
    repository: "docker.io/portkeyai/gateway_enterprise"
    pullPolicy: Always
    tag: "latest"
  dataserviceImage:
    repository: "docker.io/portkeyai/data-service"
    pullPolicy: Always
    tag: "latest"
  redisImage:
    repository: "docker.io/redis"
    pullPolicy: IfNotPresent
    tag: "7.2-alpine"
imageCredentials:
  - name: portkeyenterpriseregistrycredentials
    create: true
    registry: https://index.docker.io/v1/
    username: <DOCKER_USERNAME>
    password: <DOCKER_PASSWORD>

environment:
  create: true
  secret: true
  data:
    ANALYTICS_STORE: control_plane
    SERVICE_NAME: gateway                                                  
    PORTKEY_CLIENT_AUTH: <CLIENT_AUTH>                      # REPLACE <CLIENT_AUTH> with client auth shared by Portkey team.
    ORGANISATIONS_TO_SYNC: <ORGANIZATION_ID>                # REPLACE <ORGANIZATION_ID> with organisation_id of your account.
    PORT: "8787"

    # Configuration for using built-in redis
    CACHE_STORE: redis
    REDIS_URL: "redis://redis:6379"
    REDIS_TLS_ENABLED: "false"
   
    # Configuration for enabling IRSA access to Amazon S3
    LOG_STORE: s3_assume
    LOG_STORE_REGION: <S3_BUCKET_REGION>                    # Specify the AWS region where the S3 log bucket resides (e.g., us-east-1).
    LOG_STORE_GENERATIONS_BUCKET: <S3_BUCKET_NAME>          # Specify the name of the Amazon S3 bucket (e.g., portkey-log-store).



# Configuration for enabling Data Service
dataservice:
  name: "dataservice"
  enabled: true
  env:
    DEBUG_ENABLED: false
    SERVICE_NAME: "portkeyenterprise-dataservice"

# Enabling IRSA for providing Gateway access to Amazon S3 and, optionally Amazon Bedrock.
serviceAccount:
  create: true
  automount: true
  name: gateway-sa
  annotations:
    eks.amazonaws.com/role-arn: <IAM_ROLE_ARN>               # Specify the IAM Role ARN created for enabling IRSA access to Amazon S3 bucket


# Enabling Load Balancer to provide access outside of cluster
service:
  type: LoadBalancer
  port: 80                                                                                    
  containerPort: 8787
  annotations: 
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-internal: "true"
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"     
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: "/v1/health"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: "http" 
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "8787" 
    service.beta.kubernetes.io/aws-load-balancer-manage-backend-security-group-rules: "true"
```