Vercel AI SDK + Gonka AI — AI applications in TypeScript for pennies

JoinGonka API key: register at gate.joingonka.ai/register—we give 10M free tokens to start. In the Dashboard, create a key with the jg- prefix.

Install packages. For a custom OpenAI-compatible endpoint, Vercel AI SDK recommends the @ai-sdk/openai-compatible provider:

npm install ai @ai-sdk/openai-compatible

Minimal connection—create an instance of the provider via createOpenAICompatible and call generateText:

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { generateText } from 'ai';

const gonka = createOpenAICompatible({
  name: 'gonka',
  baseURL: 'https://gate.joingonka.ai/v1',
  apiKey: process.env.GONKA_API_KEY, // jg-your-key
});

const { text } = await generateText({
  model: gonka('Qwen/Qwen3-235B-A22B-Instruct-2507-FP8'),
  prompt: 'Explain what a decentralized inference network is',
});

console.log(text);

The apiKey parameter automatically adds the Authorization: Bearer jg-your-key header—no separate setup is needed. Store the key in an environment variable GONKA_API_KEY (e.g., in .env.local), not directly in the code.

Alternative—the @ai-sdk/openai package with the createOpenAI({ baseURL, apiKey }) factory. Both methods work; for endpoints not belonging to OpenAI, the AI SDK documentation specifically advises @ai-sdk/openai-compatible—it doesn't pull in unnecessary OpenAI-specific assumptions.

The main feature of Vercel AI SDK is streaming response delivery. The streamText function starts streaming tokens immediately, and the toUIMessageStreamResponse() helper delivers the ready stream directly from the route handler in Next.js App Router.

Server handler app/api/chat/route.ts:

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { streamText, convertToModelMessages, type UIMessage } from 'ai';

const gonka = createOpenAICompatible({
  name: 'gonka',
  baseURL: 'https://gate.joingonka.ai/v1',
  apiKey: process.env.GONKA_API_KEY,
});

// allow streaming for up to 30 seconds
export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: gonka('Qwen/Qwen3-235B-A22B-Instruct-2507-FP8'),
    system: 'You are a helpful assistant. Answer briefly and to the point.',
    messages: convertToModelMessages(messages),
    maxOutputTokens: 8192, // Qwen3-235B ceiling via Gateway
  });

  return result.toUIMessageStreamResponse();
}

On the client, connect the useChat hook from @ai-sdk/react—it automatically accesses /api/chat and renders the message stream. The backend, meanwhile, communicates with Gonka, not OpenAI.

Script without UI (Node, async-iterator over the stream):

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { streamText } from 'ai';

const gonka = createOpenAICompatible({
  name: 'gonka',
  baseURL: 'https://gate.joingonka.ai/v1',
  apiKey: process.env.GONKA_API_KEY,
});

const result = streamText({
  model: gonka('Qwen/Qwen3-235B-A22B-Instruct-2507-FP8'),
  prompt: 'Write a haiku about distributed computing',
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

Model parameters. Three models are available via the Gateway; all have 128K context tokens; the output ceiling (maxOutputTokens) varies:

• Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 — default, up to 8192 output tokens;
• moonshotai/Kimi-K2.6 — up to 3072 (Kimi K2.6);
• MiniMaxAI/MiniMax-M2.7 — up to 4096 (MiniMax M2.7).

If maxOutputTokens is not specified, for a non-stream request, the Gateway will default to returning up to 1500 tokens—for streaming chats, it's better to specify the value explicitly.

Vercel AI SDK typically powers interactive interfaces—chat, agent, in-app assistant. Each message carries the dialogue history, each agent step carries tool context. Therefore, the actual cost is calculated not per single request, but by production load. Let's compare typical scenarios:

JoinGonka's price is about $0.0005 per 1M input tokens, with output approximately three times more expensive. For an application with thousands of requests per day, this is the difference between a bill for hundreds of dollars and a bill for cents. The 10M free tokens are enough to fully run and debug a project before going into production.

Vercel AI SDK describes tools declaratively using a tools object and a zod schema. Qwen3-235B supports native function calling, so the AI SDK receives structured tool_calls without parsing text responses. The stopWhen: stepCountIs(n) parameter allows several consecutive steps—the model calls the tool, receives the result, and continues.

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { generateText, tool, stepCountIs } from 'ai';
import { z } from 'zod';

const gonka = createOpenAICompatible({
  name: 'gonka',
  baseURL: 'https://gate.joingonka.ai/v1',
  apiKey: process.env.GONKA_API_KEY,
});

const { text } = await generateText({
  model: gonka('Qwen/Qwen3-235B-A22B-Instruct-2507-FP8'),
  stopWhen: stepCountIs(5),
  tools: {
    weather: tool({
      description: 'Get the weather in a city',
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => ({ city, tempC: 17 }),
    }),
  },
  prompt: 'What's the weather like in Moscow? Answer in one sentence.',
});

console.log(text);

The model calls the weather tool, receives the result, and forms the final answer. The entire cycle costs about $0.00001 via Gonka versus $0.03-0.10 via Anthropic or OpenAI. For agent applications where each user request unfolds into 5-10 steps, the production savings amount to thousands of dollars per month.

If you're building an AI application in Python, check out the guide for LangChain—it uses the same approach via an OpenAI-compatible class.