LlamaIndex + Gonka AI — RAG applications for pennies

JoinGonka API Key: register at gate.joingonka.ai/register — we give 3M free tokens at the start — and create a jg-xxx key in Dashboard.

Installation:

pip install llama-index llama-index-llms-openai-like

For any OpenAI-compatible API, LlamaIndex provides the OpenAILike class from the llama_index.llms.openai_like package. A minimal example of a request to Gonka:

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    api_base="https://gate.joingonka.ai/v1",
    api_key="jg-your-key",
    model="MiniMaxAI/MiniMax-M2.7",
    is_chat_model=True,            # Gonka is a chat endpoint
    is_function_calling_model=True, # native tool calling is supported
    context_window=200000,         # 200K for network models
    max_tokens=8192,               # output ceiling via Gateway
)

response = llm.complete("Explain what RAG is in three sentences.")
print(response)

Important note on OpenAILike: be sure to set is_chat_model=True — otherwise LlamaIndex will go to the completion endpoint, which we don't have. is_function_calling_model=True enables native tool calls. Set context_window based on the model so LlamaIndex correctly splits the context.

A classic LlamaIndex scenario is an index over your documents and queries to it via query_engine. The global LLM is set once via Settings.llm, and the entire pipeline will use Gonka automatically.

from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    Settings,
)
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

# 1. LLM via Gonka (global setting)
Settings.llm = OpenAILike(
    api_base="https://gate.joingonka.ai/v1",
    api_key="jg-your-key",
    model="MiniMaxAI/MiniMax-M2.7",
    is_chat_model=True,
    context_window=200000,
    max_tokens=8192,
)

# 2. Local embeddings (free, no OpenAI)
Settings.embed_model = HuggingFaceEmbedding(
    model_name="BAAI/bge-small-en-v1.5"
)

# 3. Load and index documents from the ./data directory
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

# 4. Query the knowledge base
query_engine = index.as_query_engine()
response = query_engine.query("What is this document about?")
print(response)

A critical nuance regarding embeddings: by default, VectorStoreIndex uses OpenAI embeddings (text-embedding-ada-002) — these are separate paid calls to OpenAI, not Gonka. To avoid OpenAI entirely, specify a local embedding model via Settings.embed_model (as shown in the example above — HuggingFaceEmbedding, using pip install llama-index-embeddings-huggingface). In this case, generation occurs via Gonka, and vectorization happens locally and for free.

Cost: a single RAG pipeline request (search + generation) consumes ~2–5K LLM tokens. Via Gonka, this is a fraction of a cent; via OpenAI/Anthropic, it is 3–4 orders of magnitude more expensive. At a throughput of thousands of requests per day, the difference results in tens of thousands of dollars in savings per month.

A RAG application is not a one-time chat but a constant stream of requests: each user question consumes 2–5K LLM tokens (the question itself plus found context fragments). Let's calculate typical volumes and their costs across different providers. Gonka prices via JoinGonka Gateway: input ~$0.0069/1M, output ×3.

With the free 3M tokens, you can run the first RAG pipeline requests, index a test corpus, and process thousands of queries—without spending a cent. At production scale, JoinGonka Gateway turns RAG from an expensive service into an expense line you might not even notice.

LlamaIndex is capable not only of answering based on documents but also of building agents with tools. All three Gonka models support native tool calling — agents call functions in a structured way, without text parsing. Example of an agent with a tool:

import asyncio
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    api_base="https://gate.joingonka.ai/v1",
    api_key="jg-your-key",
    model="MiniMaxAI/MiniMax-M2.7",
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=200000,
    max_tokens=8192,
)

def multiply(a: float, b: float) -> float:
    """Multiplies two numbers."""
    return a * b

agent = FunctionAgent(
    tools=[multiply],
    llm=llm,
    system_prompt="You are a helpful assistant. Use tools for calculations.",
)

async def main():
    result = await agent.run("What is 1234 multiplied by 5678?")
    print(result)

asyncio.run(main())

Model Selection (model field and corresponding context_window / max_tokens limits):

The max_tokens limit via Gateway is up to 8192 for MiniMax M2.7 and GLM-5.3 Flash, and up to 32768 for DeepSeek V4 Flash. If max_tokens is not set for a non-streaming request, it defaults to up to 1500 tokens — for RAG responses and agent steps, set the value explicitly.

TypeScript: for LlamaIndex.TS, there is a mirror path — the OpenAI class from the @llamaindex/openai package accepts baseURL and apiKey (or reads the OPENAI_BASE_URL / OPENAI_API_KEY environment variables), so the same Gateway can be connected in the Node.js stack. If you are building AI applications on Python frameworks, also take a look at the guide for LangChain.