Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily.
Normally, Workers AI requires you to specify the model name in the cURL endpoint or within the env.AI.run function.
With OpenAI compatible endpoints, you can leverage the openai-node sdk ↗ to make calls to Workers AI. This allows you to use Workers AI by simply changing the base URL and the model name.
import OpenAI from "openai";
const openai = new OpenAI({
apiKey: env.CLOUDFLARE_API_KEY,
baseURL: `https://api.cloudflare.com/client/v4/accounts/${env.CLOUDFLARE_ACCOUNT_ID}/ai/v1`,
});
// Use chat completions
const chatCompletion = await openai.chat.completions.create({
messages: [{ role: "user", content: "Make some robot noises" }],
model: "@cf/meta/llama-3.1-8b-instruct",
});
// Use responses
const response = await openai.responses.create({
model: "@cf/openai/gpt-oss-120b",
input: "Talk to me about open source",
});
const embeddings = await openai.embeddings.create({
model: "@cf/baai/bge-large-en-v1.5",
input: "I love matcha",
});curl --request POST \
--url https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/v1/chat/completions \
--header "Authorization: Bearer {api_token}" \
--header "Content-Type: application/json" \
--data '
{
"model": "@cf/meta/llama-3.1-8b-instruct",
"messages": [
{
"role": "user",
"content": "how to build a wooden spoon in 3 short steps? give as short as answer as possible"
}
]
}
'For synchronous Chat Completions, set options.rejectIfBusy in the top-level request body. This makes the request fail instead of waiting in a capacity queue.
OpenAI clients that preserve custom fields can send this option. Clients that remove unknown fields do not apply it, so requests proceed normally.
Refer to Reject busy requests for examples and error behavior.
These endpoints are also compatible with AI Gateway.