Cloudflare Workers AI

Cloudflare Workers AI can be used for both chat and tab autocompletion in Continue. Here is an example of Cloudflare Workers AI configuration:

YAML

config.yaml

models:
  - name: Llama 3 8B
    provider: cloudflare
    accountId: YOUR CLOUDFLARE ACCOUNT ID
    apiKey: <YOUR_CLOUDFARE_API_KEY>
    model: "@cf/meta/llama-3-8b-instruct"
    contextLength: 2400
    defaultCompletionOptions:
      maxTokens: 500
    roles:
      - chat
  - name: DeepSeek Coder 6.7b Instruct
    provider: cloudflare
    accountId: YOUR CLOUDFLARE ACCOUNT ID
    apiKey: <YOUR_CLOUDFARE_API_KEY>
    model: "@hf/thebloke/deepseek-coder-6.7b-instruct-awq"
    contextLength: 2400
    defaultCompletionOptions:
      maxTokens: 500
    roles:
      - chat
  - name: DeepSeek 7b
    provider: cloudflare
    accountId: YOUR CLOUDFLARE ACCOUNT ID
    apiKey: <YOUR_CLOUDFARE_API_KEY>
    model: "@hf/thebloke/deepseek-coder-6.7b-base-awq"
    roles:
      - autocomplete

JSON

config.json

{
  "models": [
    {
      "title": "Llama 3 8B",
      "provider": "cloudflare",
      "accountId": "YOUR CLOUDFLARE ACCOUNT ID",
      "apiKey": "YOUR CLOUDFLARE API KEY",
      "contextLength": 2400,
      "completionOptions": {
        "maxTokens": 500
      },
      "model": "@cf/meta/llama-3-8b-instruct"
    },
    {
      "title": "DeepSeek Coder 6.7b Instruct",
      "provider": "cloudflare",
      "accountId": "YOUR CLOUDFLARE ACCOUNT ID",
      "apiKey": "YOUR CLOUDFLARE API KEY",
      "contextLength": 2400,
      "completionOptions": {
        "maxTokens": 500
      },
      "model": "@hf/thebloke/deepseek-coder-6.7b-instruct-awq"
    }
  ],
  "tabAutocompleteModel": {
    "title": "DeepSeek 7b",
    "provider": "cloudflare",
    "accountId": "YOUR CLOUDFLARE ACCOUNT ID",
    "apiKey": "YOUR CLOUDFLARE API KEY",
    "model": "@hf/thebloke/deepseek-coder-6.7b-base-awq"
  }
}

Visit the Cloudflare dashboard to create an API key.

Review available models on Workers AI

View the source