llm

Small LLM provider clients and kit-agent adapters for Kit

Files

Dependencies

• kit-agent

Development-only examples also use:

• kit-embeddings

kit-llm requires the net capability for provider calls. The embeddings example uses SQLite through kit-embeddings, so that example also needs ffi,file.

Installation

kit add gitlab.com/kit-lang/packages/kit-llm.git

Usage

import Auth.Net.{net-auth, http-auth}

import Kit.LLM as LLM

main = fn(env: Env) =>
  match Map.get env.vars "OPENAI_API_KEY"
    | Some raw-api-key ->
      if String.length raw-api-key == 0 then
        println "OPENAI_API_KEY must not be empty."
      else
        api-key: NonEmptyString = raw-api-key
        http = http-auth (net-auth env.root)
        client = LLM.openai-compatible "https://api.openai.com/v1" api-key "gpt-5-nano"
        messages = [
          LLM.system-message "Be concise.",
          LLM.user-message "Say hello from Kit."
        ]
        match LLM.chat http client messages
          | Ok text -> println text
          | Err e -> println "LLM error: ${e}"
    | None -> println "Set OPENAI_API_KEY to run this example."

main

Environment variables used by the packaged examples:

• OPENAI_API_KEY runs the OpenAI-compatible chat, agent, and embeddings examples.
• ANTHROPIC_API_KEY runs the Anthropic Messages API example.

Kit.LLM.Agent adapts provider chat responses to Agent.Model. Use LLMAgent.text-model for OpenAI-compatible plain text responses, LLMAgent.anthropic-text-model for Anthropic text responses, and LLMAgent.model when an OpenAI-compatible provider may return tool calls.

Public constructors and setters use Kit Refinements where the package can make bad requests harder to build:

• API keys, model names, and base URLs are NonEmptyString.
• Anthropic max-tokens and OpenAI-compatible with-max-tokens use PositiveInt.
• OpenAI-compatible temperatures use NonNegativeFloat and are rejected above 2.0.
• Chat rejects empty provider message lists before making HTTP calls.
• Embeddings reject empty input before making HTTP calls.
• Response parsers return Result errors instead of assuming provider JSON has the expected text or vector shape.

Development

Running Examples

Run the OpenAI-compatible chat example with the interpreter:

kit run --allow=net examples/basic.kit

Compile the OpenAI-compatible chat example to a native binary:

kit build --allow=net examples/basic.kit && ./basic

Run the Anthropic example:

kit run --allow=net examples/anthropic.kit

Run the streaming parser example:

kit run examples/streaming.kit

Run the embeddings example:

kit run --allow=net,ffi,file examples/embeddings.kit

Running Tests

Run the test suite:

kit test

Run the test suite with coverage:

kit test --coverage

Running kit dev

Run the standard development workflow (format, check, test):

kit dev

This will:

1. Format and check source files in src/
2. Type check examples in examples/
3. Run tests in tests/ with coverage

Generating Documentation

Generate API documentation from doc comments:

kit doc

Note: Kit sources with doc comments (##) will generate HTML documents in docs/*.html

Cleaning Build Artifacts

Remove generated files, caches, and build artifacts:

kit task clean

Note: Defined in kit.toml.

Local Installation

To install this package locally for development:

kit install

This installs the package to ~/.kit/packages/@kit/llm/, making it available for import as Kit.LLM in other projects.

License

This package is released under the MIT License - see LICENSE for details.

Exported Functions & Types

StreamEvent

openai-compatible

Create an OpenAI-compatible client configuration.

with-header

Add a provider-specific request header.

with-temperature

Set temperature for providers that use the OpenAI 0.0 to 2.0 range.

with-max-tokens

Set the maximum generated tokens.

system-message

Create a system chat message.

user-message

Create a user chat message.

assistant-message

Create an assistant chat message.

stream-state

Create an empty incremental stream parser state.

chat-url

Build the OpenAI-compatible chat completions URL.

embedding-url

Build the OpenAI-compatible embeddings URL.

build-chat-request-json

Build the JSON request body value for a chat completion.

build-chat-request

Build the JSON request body for a chat completion.

build-embedding-request

Build the JSON request body for a text embedding.

parse-chat-response

Parse an OpenAI-compatible chat response body into assistant text.

provider-error-message

Convert a non-2xx provider response into a concise error.

parse-embedding-response

Parse an OpenAI-compatible embeddings response body into one vector.

chat-response

Run a chat request body and return the raw successful provider response.

embedding-response

Run an embeddings request body and return the raw successful provider response.

chat

Run one non-streaming OpenAI-compatible chat completion.

embed

Generate one text embedding from an OpenAI-compatible endpoint.

parse-stream-chunk

Parse one incoming OpenAI-compatible SSE text chunk.

parse-stream-events

Parse a complete OpenAI-compatible SSE body into stream events.

messages-for-context

Build chat messages from a kit-agent runner context.

text-model-with-chat

Create a text-only Agent.Model from a custom chat function.

text-model

Create a text-only Agent.Model backed by an OpenAI-compatible client.

anthropic-text-model

Create a text-only Agent.Model backed by an Anthropic Messages API client.

build-chat-request

Build a chat request body with provider-facing tool declarations.

parse-chat-action-response

Parse an OpenAI-compatible chat response into an Agent.Action.

model-with-chat-response

Create an Agent.Model from a custom raw chat response function.

model

Create an Agent.Model backed by an OpenAI-compatible client with tool-call support.

tool-declaration-json

Convert an Agent.ToolDeclaration to an OpenAI-compatible tool schema.

tool-declarations-json

Convert Agent.ToolDeclaration values to OpenAI-compatible tool schemas.

client

Create an Anthropic direct API client configuration.

with-header

Add a provider-specific request header.

messages-url

Build the Anthropic Messages API URL.

build-messages-request-json

Build the JSON request body value for an Anthropic message.

build-messages-request

Build the JSON request body for an Anthropic message.

parse-message-response

Parse an Anthropic Messages API response body into assistant text.

provider-error-message

Convert a non-2xx Anthropic response into a concise error.

message-response

Run an Anthropic request body and return the raw successful provider response.

chat

Run one non-streaming Anthropic chat request.