> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-feature-comparison-update.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Predibase

> Use Predibase's open-source and fine-tuned LLMs through Portkey.

## Quick Start

Get started with Predibase in under 2 minutes:

<CodeGroup>
  ```python Python icon="python" theme={null}
  from portkey_ai import Portkey

  # 1. Install: pip install portkey-ai
  # 2. Add @predibase provider in model catalog
  # 3. Use it:

  portkey = Portkey(api_key="PORTKEY_API_KEY")

  response = portkey.chat.completions.create(
      model="@predibase/llama-3-8b-instruct",
      messages=[{"role": "user", "content": "Hello!"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js Javascript icon="square-js" theme={null}
  import Portkey from 'portkey-ai'

  // 1. Install: npm install portkey-ai
  // 2. Add @predibase provider in model catalog
  // 3. Use it:

  const portkey = new Portkey({
      apiKey: "PORTKEY_API_KEY"
  })

  const response = await portkey.chat.completions.create({
      model: "@predibase/llama-3-8b-instruct",
      messages: [{ role: "user", content: "Hello!" }]
  })

  console.log(response.choices[0].message.content)
  ```

  ```python OpenAI Py icon="python" theme={null}
  from openai import OpenAI
  from portkey_ai import PORTKEY_GATEWAY_URL

  # 1. Install: pip install openai portkey-ai
  # 2. Add @predibase provider in model catalog
  # 3. Use it:

  client = OpenAI(
      api_key="PORTKEY_API_KEY",  # Portkey API key
      base_url=PORTKEY_GATEWAY_URL
  )

  response = client.chat.completions.create(
      model="@predibase/llama-3-8b-instruct",
      messages=[{"role": "user", "content": "Hello!"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js OpenAI JS icon="square-js" theme={null}
  import OpenAI from "openai"
  import { PORTKEY_GATEWAY_URL } from "portkey-ai"

  // 1. Install: npm install openai portkey-ai
  // 2. Add @predibase provider in model catalog
  // 3. Use it:

  const client = new OpenAI({
      apiKey: "PORTKEY_API_KEY",  // Portkey API key
      baseURL: PORTKEY_GATEWAY_URL
  })

  const response = await client.chat.completions.create({
      model: "@predibase/llama-3-8b-instruct",
      messages: [{ role: "user", content: "Hello!" }]
  })

  console.log(response.choices[0].message.content)
  ```

  ```sh cURL icon="square-terminal" theme={null}
  # 1. Add @predibase provider in model catalog
  # 2. Use it:

  curl https://api.portkey.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -d '{
      "model": "@predibase/llama-3-8b-instruct",
      "messages": [{"role": "user", "content": "Hello!"}]
    }'
  ```
</CodeGroup>

## Add Provider in Model Catalog

Before making requests, add Predibase to your Model Catalog:

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Predibase**
3. Enter your [Predibase API key](https://app.predibase.com/settings)
4. Name your provider (e.g., `predibase`)

<Card title="Complete Setup Guide" icon="book" href="/product/model-catalog">
  See all setup options and detailed configuration instructions
</Card>

***

## Predibase Capabilities

### Serverless Endpoints

Predibase offers LLMs like **Llama 3**, **Mistral**, **Gemma**, etc. on its [serverless infrastructure](https://docs.predibase.com/user-guide/inference/models#serverless-endpoints) that you can query instantly.

<Note>
  **Sending Predibase Tenant ID**

  Predibase expects your **account tenant ID** along with the API key in each request. With Portkey, you can send [**your Tenant ID**](https://app.predibase.com/settings) with the `user` param while making your request.
</Note>

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey

  portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@predibase")

  response = portkey.chat.completions.create(
      model="llama-3-8b-instruct",
      messages=[{"role": "user", "content": "Hello!"}],
      user="PREDIBASE_TENANT_ID"  # Required: Your Predibase tenant ID
  )

  print(response.choices[0].message.content)
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY',
      provider: '@predibase'
  });

  const response = await portkey.chat.completions.create({
      model: "llama-3-8b-instruct",
      messages: [{ role: "user", content: "Hello!" }],
      user: "PREDIBASE_TENANT_ID"  // Required: Your Predibase tenant ID
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

### Using Fine-Tuned Models

Predibase allows you to deploy and use fine-tuned models with adapters. Use the special format with your model identifier from your Predibase dashboard:

<Note>
  **Fine-Tuned Model Format:**\
  `model = base_model:adapter-repo-name/adapter-version-number`

  For example: `llama-3-8b:sentiment-analysis/1`
</Note>

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey

  portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@predibase")

  response = portkey.chat.completions.create(
      model="llama-3-8b:sentiment-analysis/1",  # Base model + adapter
      messages=[{"role": "user", "content": "This product is amazing!"}],
      user="PREDIBASE_TENANT_ID"
  )

  print(response.choices[0].message.content)
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY',
      provider: '@predibase'
  });

  const response = await portkey.chat.completions.create({
      model: "llama-3-8b:sentiment-analysis/1",  // Base model + adapter
      messages: [{ role: "user", content: "This product is amazing!" }],
      user: "PREDIBASE_TENANT_ID"
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

### Dedicated Deployments

Route requests to your dedicated deployed models by passing the deployment name in the `model` parameter:

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey

  portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@predibase")

  response = portkey.chat.completions.create(
      model="my-dedicated-mistral-deployment",  # Your deployment name
      messages=[{"role": "user", "content": "Hello!"}],
      user="PREDIBASE_TENANT_ID"
  )

  print(response.choices[0].message.content)
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY',
      provider: '@predibase'
  });

  const response = await portkey.chat.completions.create({
      model: "my-dedicated-mistral-deployment",  // Your deployment name
      messages: [{ role: "user", content: "Hello!" }],
      user: "PREDIBASE_TENANT_ID"
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

### JSON Schema Mode

Enforce JSON schema for all Predibase models by setting `response_format` to `json_object` with your schema:

<CodeGroup>
  ```python Python theme={null}
  from portkey_ai import Portkey
  from pydantic import BaseModel, constr

  # Define JSON Schema with Pydantic
  class Character(BaseModel):
      name: constr(max_length=10)
      age: int
      strength: int

  portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@predibase")

  response = portkey.chat.completions.create(
      model="llama-3-8b",
      messages=[{"role": "user", "content": "Create a character profile"}],
      user="PREDIBASE_TENANT_ID",
      response_format={
          "type": "json_object",
          "schema": Character.schema()
      }
  )

  print(response.choices[0].message.content)
  ```

  ```javascript Node.js theme={null}
  import Portkey from 'portkey-ai';

  const portkey = new Portkey({
      apiKey: 'PORTKEY_API_KEY',
      provider: '@predibase'
  });

  const response = await portkey.chat.completions.create({
      model: "llama-3-8b",
      messages: [{ role: "user", content: "Create a character profile" }],
      user: "PREDIBASE_TENANT_ID",
      response_format: {
          type: "json_object",
          schema: {
              properties: {
                  name: { maxLength: 10, title: "Name", type: "string" },
                  age: { title: "Age", type: "integer" },
                  strength: { title: "Strength", type: "integer" }
              },
              required: ["name", "age", "strength"],
              title: "Character",
              type: "object"
          }
      }
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

***

## Supported Models

Predibase provides access to various open-source and fine-tuned models:

* Llama 3 (various sizes)
* Mistral
* Zephyr
* Your custom fine-tuned models

Check [Predibase's documentation](https://docs.predibase.com/) for the complete model list and fine-tuning options.

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Gateway Configs" icon="sliders" href="/product/ai-gateway">
    Add fallbacks, load balancing, and more
  </Card>

  <Card title="Observability" icon="chart-line" href="/product/observability">
    Monitor and trace your Predibase requests
  </Card>

  <Card title="Prompt Library" icon="book" href="/product/prompt-engineering-studio">
    Manage and version your prompts
  </Card>

  <Card title="Fine-Tuning Logs" icon="chart-mixed" href="/product/observability">
    Track performance of fine-tuned models
  </Card>
</CardGroup>

For complete SDK documentation:

<Card title="SDK Reference" icon="code" href="/api-reference/sdk/list">
  Complete Portkey SDK documentation
</Card>