LlamaIndex + Gonka AI — RAG applications for pennies

JoinGonka API key: sign up at gate.joingonka.ai/register — you get 3M free tokens to start — and create a jg-xxx key in the Dashboard.

Installation:

pip install llama-index llama-index-llms-openai-like

For any OpenAI-compatible API, LlamaIndex provides the OpenAILike class from the llama_index.llms.openai_like package. Here's a minimal request to Gonka:

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    api_base="https://gate.joingonka.ai/v1",
    api_key="jg-your-key",
    model="MiniMaxAI/MiniMax-M2.7",
    is_chat_model=True,            # Gonka is a chat endpoint
    is_function_calling_model=True, # native tool calling is supported
    context_window=180000,         # MiniMax M2.7 context window
    max_tokens=8192,               # Gateway output cap
)

response = llm.complete("Explain what RAG is in three sentences.")
print(response)

Important about OpenAILike: always set is_chat_model=True — otherwise LlamaIndex will hit the legacy Completions endpoint with a raw prompt instead of a chat. is_function_calling_model=True enables native tool calls. Set context_window per model so LlamaIndex chunks context correctly.

The classic LlamaIndex scenario — an index over your documents with queries against it through query_engine. The global LLM is set once via Settings.llm, and from then on the whole pipeline uses Gonka automatically.

from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    Settings,
)
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

# 1. LLM via Gonka (set once, globally)
Settings.llm = OpenAILike(
    api_base="https://gate.joingonka.ai/v1",
    api_key="jg-your-key",
    model="MiniMaxAI/MiniMax-M2.7",
    is_chat_model=True,
    context_window=180000,
    max_tokens=8192,
)

# 2. Local embeddings (free, no OpenAI)
Settings.embed_model = HuggingFaceEmbedding(
    model_name="BAAI/bge-small-en-v1.5"
)

# 3. Load and index documents from the ./data folder
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

# 4. Query the knowledge base
query_engine = index.as_query_engine()
response = query_engine.query("What is this document about?")
print(response)

A critical caveat about embeddings: by default VectorStoreIndex uses OpenAI embeddings (text-embedding-ada-002) — those are separate paid calls to OpenAI, not to Gonka. To move fully off OpenAI, set a local embedding model via Settings.embed_model (as in the example above — HuggingFaceEmbedding, package pip install llama-index-embeddings-huggingface). Then generation runs through Gonka while vectorization happens locally and for free.

Cost: a single RAG pipeline request (retrieval + generation) consumes ~2–5K LLM tokens. Through Gonka that's fractions of a cent; through OpenAI/Anthropic it's 3–4 orders of magnitude more expensive. At a volume of thousands of requests per day, the difference turns into tens of thousands of dollars in monthly savings.

A RAG application isn't a one-off chat, it's a constant stream of requests: every user question pulls in 2–5K LLM tokens (the question itself plus the retrieved context fragments). Let's tally up typical volumes and what they cost across different providers. Gonka prices through JoinGonka Gateway: input ~$0.0062/1M, output ×3.

With the free 3M tokens, you can run your first RAG pipeline requests, index a test corpus, and push through thousands of queries — without spending a cent. At production volumes, JoinGonka Gateway turns RAG from an expensive service into a line item you barely even notice.

LlamaIndex can do more than answer questions over documents — it can also build agents with tools. All three Gonka models support native tool calling: agents invoke functions in a structured way, with no text parsing. Here's an example of an agent with a tool:

import asyncio
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    api_base="https://gate.joingonka.ai/v1",
    api_key="jg-your-key",
    model="MiniMaxAI/MiniMax-M2.7",
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=180000,
    max_tokens=8192,
)

def multiply(a: float, b: float) -> float:
    """Multiplies two numbers."""
    return a * b

agent = FunctionAgent(
    tools=[multiply],
    llm=llm,
    system_prompt="You are a helpful assistant. Use the tools for calculations.",
)

async def main():
    result = await agent.run("What is 1234 multiplied by 5678?")
    print(result)

asyncio.run(main())

Choosing a model (the model field and the corresponding context_window / max_tokens limits):

The max_tokens limit through the Gateway is up to 8192 for MiniMax M2.7 and GLM-5.3 Flash, and up to 32768 for DeepSeek V4 Flash. If max_tokens is not set for a non-streaming request, up to 1500 tokens are returned by default — for RAG answers and agent steps, set the value explicitly.

TypeScript: LlamaIndex.TS offers a mirror path — the OpenAI class from the @llamaindex/openai package accepts baseURL and apiKey (or reads the OPENAI_BASE_URL / OPENAI_API_KEY environment variables), so the same Gateway plugs into a Node.js stack too. If you're building AI applications on Python frameworks as well, check out the LangChain guide too.