LlamaIndex + Gonka AI — RAG applications for pennies

JoinGonka API key: register at gate.joingonka.ai/register — we give you 10M free tokens to get started — and create a jg-xxx key in the Dashboard.

Installation:

pip install llama-index llama-index-llms-openai-like

For any arbitrary OpenAI-compatible API, LlamaIndex provides the OpenAILike class from the llama_index.llms.openai_like package. A minimal request example to Gonka:

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    api_base="https://gate.joingonka.ai/v1",
    api_key="jg-your-key",
    model="MiniMaxAI/MiniMax-M2.7",
    is_chat_model=True,            # Gonka is a chat endpoint
    is_function_calling_model=True, # native tool calling is supported
    context_window=200000,         # 200K for network models
    max_tokens=8192,               # output ceiling via Gateway
)

response = llm.complete("Explain what RAG is in three sentences.")
print(response)

Important note on OpenAILike: be sure to specify is_chat_model=True — otherwise, LlamaIndex will attempt to use the completion endpoint, which we do not have. is_function_calling_model=True enables native tool calls. Set context_window according to the model so that LlamaIndex truncates context correctly.

A classic LlamaIndex scenario is an index over your documents and queries to it via query_engine. The global LLM is set once via Settings.llm, and the entire pipeline will use Gonka automatically.

from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    Settings,
)
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

# 1. LLM via Gonka (global setting)
Settings.llm = OpenAILike(
    api_base="https://gate.joingonka.ai/v1",
    api_key="jg-your-key",
    model="MiniMaxAI/MiniMax-M2.7",
    is_chat_model=True,
    context_window=200000,
    max_tokens=8192,
)

# 2. Local embeddings (free, no OpenAI)
Settings.embed_model = HuggingFaceEmbedding(
    model_name="BAAI/bge-small-en-v1.5"
)

# 3. Load and index documents from the ./data directory
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

# 4. Query the knowledge base
query_engine = index.as_query_engine()
response = query_engine.query("What is this document about?")
print(response)

A critical nuance regarding embeddings: by default, VectorStoreIndex uses OpenAI embeddings (text-embedding-ada-002) — these are separate paid calls to OpenAI, not Gonka. To avoid OpenAI entirely, specify a local embedding model via Settings.embed_model (as shown in the example above — HuggingFaceEmbedding, using pip install llama-index-embeddings-huggingface). In this case, generation occurs via Gonka, and vectorization happens locally and for free.

Cost: a single RAG pipeline request (search + generation) consumes ~2–5K LLM tokens. Via Gonka, this is a fraction of a cent; via OpenAI/Anthropic, it is 3–4 orders of magnitude more expensive. At a throughput of thousands of requests per day, the difference results in tens of thousands of dollars in savings per month.

A RAG application is not a one-time chat but a continuous stream of requests: every user question pulls in 2–5K LLM tokens (the question itself plus the found context fragments). Let's calculate typical volumes and costs across different providers. Gonka prices via JoinGonka Gateway: input ~$0.003/1M, output ×3.

With 10M free tokens, you can debug your entire RAG pipeline, index a test corpus, and run thousands of queries—all without spending a cent. At production scale, JoinGonka Gateway turns RAG from an expensive service into a negligible expense line item.

LlamaIndex is capable not only of answering based on documents but also building agents with tools. Both Gonka models support native tool calling — agents call functions in a structured manner without text parsing. Example of an agent with a tool:

import asyncio
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    api_base="https://gate.joingonka.ai/v1",
    api_key="jg-your-key",
    model="MiniMaxAI/MiniMax-M2.7",
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=200000,
    max_tokens=8192,
)

def multiply(a: float, b: float) -> float:
    """Multiplies two numbers."""
    return a * b

agent = FunctionAgent(
    tools=[multiply],
    llm=llm,
    system_prompt="You are a helpful assistant. Use tools for calculations.",
)

async def main():
    result = await agent.run("How much is 1234 multiplied by 5678?")
    print(result)

asyncio.run(main())

Model selection (model field and corresponding context_window / max_tokens limits):

The max_tokens limit via Gateway is up to 8192 for all network models. If max_tokens is not specified for a non-streaming request, it defaults to 1500 tokens — for RAG responses and agent steps, set this value explicitly.

TypeScript: for LlamaIndex.TS there is a mirror path — the OpenAI class from the @llamaindex/openai package accepts baseURL and apiKey (or reads the OPENAI_BASE_URL / OPENAI_API_KEY environment variables), so the same Gateway can be connected in the Node.js stack. If you are building AI applications using Python frameworks, also check out our LangChain guide.