> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pioneer.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Fine-tune an open-source LLM on Pioneer — SFT, GRPO, and DPO

> Train a custom large language model with LoRA fine-tuning on Qwen, Llama, DeepSeek, or Gemma base models. Pioneer supports supervised fine-tuning (SFT) and the reinforcement-learning methods GRPO and DPO through a single training endpoint.

Pioneer supports parameter-efficient (LoRA) post-training on a wide range of open-source decoder models — from compact 1B-parameter models to 70B+ frontier models. You bring your training data, choose a base model that fits your task and budget, and Pioneer handles the infrastructure, routing, and serving. The result is a fine-tuned adapter you can call over the same API, with no GPU management required.

All three post-training algorithms — supervised fine-tuning (`sft`), GRPO (`grpo`), and DPO (`dpo`) — are created through the same [`POST /felix/training-jobs`](/api-reference/training-jobs) endpoint and selected with the **`training_algorithm`** parameter.

## Choose an algorithm

| Algorithm                  | `training_algorithm` | What it optimizes                            | Dataset signal                            |
| -------------------------- | -------------------- | -------------------------------------------- | ----------------------------------------- |
| **Supervised fine-tuning** | `sft` *(default)*    | Imitate the assistant turns in your examples | Chat `messages`                           |
| **GRPO**                   | `grpo`               | Maximize a reward over sampled completions   | `prompt` + `answer` (+ a reward function) |
| **DPO**                    | `dpo`                | Prefer chosen responses over rejected ones   | `prompt` + `chosen` + `rejected`          |

Omitting `training_algorithm` is equivalent to `sft`, so existing requests keep working unchanged.

<CardGroup cols={3}>
  <Card title="SFT" icon="graduation-cap">
    You have example outputs you want the model to imitate (conversations, instruction-response pairs). The default and simplest path.
  </Card>

  <Card title="GRPO" icon="trophy">
    "Good" is a programmatic check — exact answers, numeric correctness, JSON validity, a rubric. The model explores and is reinforced toward higher reward.
  </Card>

  <Card title="DPO" icon="scale-balanced">
    You have preference pairs — a better and a worse response per prompt — rather than a single gold answer.
  </Card>
</CardGroup>

<Note>
  All three algorithms are **LoRA-based**. A completed job produces a low-rank adapter that is hot-swapped onto the shared base model at serve time and exposed behind the same inference endpoints as base models — reference the training job's `id` as the `model_id` at inference time. `training_type` defaults to `"lora"` and is the only supported value for decoder LLMs; `"full"` is reserved for [GLiNER encoder models](/guides/fine-tune-ner).
</Note>

## End-to-end walkthrough

<Steps>
  <Step title="Choose a decoder base model">
    Use `GET /base-models` to see the full current catalog, filtered to models that support training:

    <CodeGroup>
      ```bash cURL theme={null}
      curl "https://api.pioneer.ai/base-models?task_type=decoder&supports_training=true" \
        -H "X-API-Key: YOUR_API_KEY"
      ```
    </CodeGroup>

    The table below shows a selection of popular options. Context window size matters if your training examples or inference prompts are long.

    | Model ID                            | Label                  | Context |
    | ----------------------------------- | ---------------------- | ------- |
    | `Qwen/Qwen3-32B`                    | Qwen3 32B              | 131K    |
    | `Qwen/Qwen3-30B-A3B-Instruct-2507`  | Qwen3 30B A3B Instruct | 262K    |
    | `Qwen/Qwen3-8B`                     | Qwen3 8B               | 131K    |
    | `Qwen/Qwen3-4B-Instruct-2507`       | Qwen3 4B Instruct      | 262K    |
    | `Qwen/Qwen2.5-7B-Instruct`          | Qwen2.5 7B Instruct    | 131K    |
    | `Qwen/Qwen2.5-14B-Instruct`         | Qwen2.5 14B Instruct   | 131K    |
    | `meta-llama/Llama-3.3-70B-Instruct` | Llama 3.3 70B Instruct | 131K    |
    | `meta-llama/Llama-3.1-8B-Instruct`  | Llama 3.1 8B Instruct  | 131K    |
    | `meta-llama/Llama-3.1-70B-Instruct` | Llama 3.1 70B Instruct | 131K    |
    | `meta-llama/Llama-3.2-3B-Instruct`  | Llama 3.2 3B Instruct  | 131K    |
    | `deepseek-ai/DeepSeek-V3.1`         | DeepSeek V3.1          | 163K    |
    | `google/gemma-4-31b-it`             | Gemma 4 31B IT         | 128K    |
    | `openai/gpt-oss-120b`               | GPT-OSS 120B           | 131K    |

    **Choosing a model size:** Smaller models (1B–8B) train and respond faster and cost less. Larger models (30B–70B) handle complex reasoning and longer inputs more reliably. Start with `Qwen/Qwen3-8B` or `meta-llama/Llama-3.1-8B-Instruct` for most tasks and scale up if needed.

    Not every model supports every algorithm — see [Supported models](#supported-models) below for the SFT/GRPO/DPO matrix.
  </Step>

  <Step title="Prepare your training data">
    The dataset shape depends on the algorithm you picked. Pick the matching tab.

    <CodeGroup>
      ```bash SFT (chat messages) theme={null}
      # Each row: {"messages": [{"role": "user" | "assistant" | "system", "content": "..."}]}
      # Generate synthetically:
      curl -X POST https://api.pioneer.ai/generate \
        -H "X-API-Key: YOUR_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "task_type": "decoder",
          "dataset_name": "my-sft-dataset",
          "num_examples": 200,
          "domain_description": "Customer support for a SaaS product"
        }'
      ```

      ```bash GRPO (prompt + answer) theme={null}
      # Each row: {"prompt": "...", "answer": "..."}
      # The answer is the per-row ground truth read by the reward function.
      # Upload through the Pioneer dashboard, then confirm:
      curl https://api.pioneer.ai/felix/datasets/my-grpo-dataset \
        -H "X-API-Key: YOUR_API_KEY"
      ```

      ```bash DPO (preference pairs) theme={null}
      # Each row: {"prompt": "...", "chosen": "...", "rejected": "..."}
      # Upload through the Pioneer dashboard, then confirm:
      curl https://api.pioneer.ai/felix/datasets/my-dpo-dataset \
        -H "X-API-Key: YOUR_API_KEY"
      ```
    </CodeGroup>

    See the [Synthetic Data guide](/guides/synthetic-data) for the full set of `/generate` options, including auto-labelling existing text. Once generated or uploaded, wait until the dataset status is `ready` before starting training.
  </Step>

  <Step title="Start a training job">
    Submit your training job with `POST /felix/training-jobs`. The `training_algorithm` parameter selects SFT, GRPO, or DPO.

    <CodeGroup>
      ```bash SFT theme={null}
      curl -X POST https://api.pioneer.ai/felix/training-jobs \
        -H "X-API-Key: YOUR_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "model_name": "my-sft-model",
          "base_model": "Qwen/Qwen3-8B",
          "training_type": "lora",
          "datasets": [{"name": "my-sft-dataset", "version": "1"}],
          "lora_r": 16,
          "lora_alpha": 32,
          "learning_rate": 2e-5,
          "nr_epochs": 3
        }'
      ```

      ```bash GRPO theme={null}
      curl -X POST https://api.pioneer.ai/felix/training-jobs \
        -H "X-API-Key: YOUR_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "model_name": "my-grpo-model",
          "base_model": "Qwen/Qwen3-4B-Instruct-2507",
          "training_type": "lora",
          "training_algorithm": "grpo",
          "datasets": [{"name": "gsm8k-grpo", "version": "1"}],
          "rl_config": {
            "reward_type": "numeric_match",
            "max_steps": 100,
            "group_size": 8
          }
        }'
      ```

      ```bash DPO theme={null}
      curl -X POST https://api.pioneer.ai/felix/training-jobs \
        -H "X-API-Key: YOUR_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "model_name": "my-dpo-model",
          "base_model": "meta-llama/Llama-3.1-8B-Instruct",
          "training_type": "lora",
          "training_algorithm": "dpo",
          "datasets": [{"name": "my-preferences", "version": "1"}],
          "rl_config": {
            "dpo_beta": 0.1
          }
        }'
      ```
    </CodeGroup>

    Pioneer routes your job automatically to the best available provider. The response includes your job ID:

    ```json theme={null}
    { "id": "uuid-of-training-job", "status": "requested" }
    ```

    `rl_config` is **required when** `training_algorithm` is `grpo` or `dpo` and **must be omitted for** `sft`. Every key inside `rl_config` is optional and falls back to a TRL-aligned server default except `reward_type`, which is required for GRPO.
  </Step>

  <Step title="Poll until training is complete">
    Check job status by polling `GET /felix/training-jobs/:id`.

    <CodeGroup>
      ```bash cURL theme={null}
      curl https://api.pioneer.ai/felix/training-jobs/YOUR_JOB_ID \
        -H "X-API-Key: YOUR_API_KEY"
      ```
    </CodeGroup>

    Status transitions: `requested` → `running` → `complete` → `deployed` (or `failed` / `stopped`). The terminal success state is `deployed`, reached automatically once the adapter is live behind the inference endpoints.

    You can also stream training logs while the job is running:

    <CodeGroup>
      ```bash cURL theme={null}
      curl https://api.pioneer.ai/felix/training-jobs/YOUR_JOB_ID/logs \
        -H "X-API-Key: YOUR_API_KEY"
      ```
    </CodeGroup>
  </Step>

  <Step title="Run inference on your fine-tuned model">
    Once the job status is `deployed`, use your job ID as the `model_id` (or `model`) on any of the three inference interfaces.

    **Pioneer native API** — use `"task": "generate"` for decoder models:

    <CodeGroup>
      ```bash cURL theme={null}
      curl -X POST https://api.pioneer.ai/inference \
        -H "X-API-Key: YOUR_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "model_id": "YOUR_JOB_ID",
          "task": "generate",
          "messages": [{"role": "user", "content": "Summarize this article: ..."}]
        }'
      ```
    </CodeGroup>

    **OpenAI-compatible endpoint** — drop-in replacement for the OpenAI SDK:

    <CodeGroup>
      ```python Python (OpenAI SDK) theme={null}
      from openai import OpenAI

      client = OpenAI(
          api_key="YOUR_API_KEY",
          base_url="https://api.pioneer.ai/v1"
      )

      response = client.chat.completions.create(
          model="YOUR_JOB_ID",
          messages=[{"role": "user", "content": "Summarize this article: ..."}]
      )
      print(response.choices[0].message.content)
      ```

      ```bash cURL (OpenAI-compatible) theme={null}
      curl -X POST https://api.pioneer.ai/v1/chat/completions \
        -H "X-API-Key: YOUR_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "model": "YOUR_JOB_ID",
          "messages": [{"role": "user", "content": "Summarize this article: ..."}]
        }'
      ```
    </CodeGroup>

    **Anthropic-compatible endpoint:**

    <CodeGroup>
      ```bash cURL (Anthropic-compatible) theme={null}
      curl -X POST https://api.pioneer.ai/v1/messages \
        -H "X-API-Key: YOUR_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "model": "YOUR_JOB_ID",
          "max_tokens": 1024,
          "messages": [{"role": "user", "content": "Summarize this article: ..."}]
        }'
      ```
    </CodeGroup>

    Streaming is supported on all three interfaces.
  </Step>
</Steps>

<Note>
  Downloading your trained model weights is available on the Pro plan and above. Use `GET /felix/training-jobs/:id/download` to retrieve the weights once training is complete.
</Note>

## LoRA hyperparameters

LoRA capacity and the core optimization settings are configurable; the defaults are sensible starting points for SFT and the RL algorithms alike.

| Field                        | Default | Purpose                                                                     |
| ---------------------------- | ------- | --------------------------------------------------------------------------- |
| `lora_r`                     | `16`    | LoRA rank — adapter capacity. Raise it for harder tasks or larger datasets. |
| `lora_alpha`                 | `32`    | LoRA scaling factor (typically \~2× `lora_r`).                              |
| `lora_dropout`               | `0.1`   | Dropout applied to the adapter during training.                             |
| `learning_rate`              | `2e-5`  | Peak AdamW learning rate.                                                   |
| `batch_size`                 | `4`     | Per-step batch size.                                                        |
| `nr_epochs`                  | `100`   | Epoch ceiling; early stopping usually halts well before this.               |
| `validation_data_percentage` | `0.2`   | Fraction of the dataset held out for validation.                            |

## GRPO reward functions

GRPO (Group Relative Policy Optimization) samples multiple completions per prompt and reinforces the ones that score highest against a reward function. Set `rl_config.reward_type` to one of:

| `reward_type`        | Scores a completion as correct when…                                                        |
| -------------------- | ------------------------------------------------------------------------------------------- |
| `exact_match`        | the normalized completion equals `answer`                                                   |
| `contains_substring` | `answer` appears anywhere in the completion                                                 |
| `numeric_match`      | the extracted number matches `answer` (handles `#### 42`, `\boxed{42}`, "the answer is 42") |
| `choice_match`       | the final multiple-choice letter matches `answer`                                           |
| `regex_match`        | the completion matches the supplied pattern                                                 |
| `json_match`         | the parsed JSON deep-equals `answer`                                                        |
| `json_loose_match`   | the parsed JSON loosely matches `answer`                                                    |
| `rougeL_match`       | ROUGE-L against the `answer` reference(s)                                                   |
| `llm_as_judge`       | a judge model scores the completion against a rubric                                        |

<Note>
  When `reward_type` is `llm_as_judge`, Pioneer mints and manages the judge credential for you — you never supply an API key. Optional judge knobs include `llm_judge_model`, `llm_judge_rubric`, and `llm_judge_score_scale`.
</Note>

## Supported models

The canonical, live list is always `GET /base-models?supports_training=true`. As of this writing:

| Base model                         | SFT | GRPO | DPO |
| ---------------------------------- | :-: | :--: | :-: |
| `Qwen/Qwen3-8B`                    |  ✅  |   ✅  |  ✅  |
| `Qwen/Qwen3-32B`                   |  ✅  |   ✅  |  ✅  |
| `Qwen/Qwen3-4B-Instruct-2507`      |  ✅  |   ✅  |  ✅  |
| `Qwen/Qwen3-4B-Base`               |  ✅  |   ✅  |  ✅  |
| `Qwen/Qwen3-1.7B-Base`             |  ✅  |   ✅  |  ✅  |
| `meta-llama/Llama-3.1-8B-Instruct` |  ✅  |   ✅  |  ✅  |
| `HuggingFaceTB/SmolLM3-3B-Base`    |  ✅  |   ✅  |  ✅  |
| `google/gemma-4-31b-it`            |  ✅  |   —  |  —  |
| `meta-llama/Llama-3.2-3B-Instruct` |  ✅  |   —  |  —  |
| `Qwen/Qwen2.5-7B-Instruct`         |  ✅  |   —  |  —  |

<Note>
  GRPO and DPO are available on the subset of models that have been verified end-to-end for reinforcement learning. Every trainable decoder supports SFT. Models marked `—` for RL accept `sft` only; submitting `grpo`/`dpo` for them returns a `422`.
</Note>

GLiNER encoder models (`fastino/gliner2-base-v1`, `fastino/gliner2-large-v1`, `fastino/gliner2-multi-v1`, `fastino/gliner2-multi-large-v1`) are also trainable through the same endpoint — see the encoder fine-tuning guides for [NER](/guides/fine-tune-ner), [classification](/guides/fine-tune-classification), and [structured extraction](/guides/fine-tune-extraction).

## Serverless inference for base models

If you want to run inference on a base model without fine-tuning, several models are available as serverless endpoints with no startup latency:

| Model ID                             | Label                    | Context |
| ------------------------------------ | ------------------------ | ------- |
| `Qwen/Qwen3-235B-A22B-Instruct-2507` | Qwen3 235B A22B Instruct | 262K    |
| `Qwen/Qwen3-8B`                      | Qwen3 8B                 | 131K    |
| `deepseek-ai/DeepSeek-V3.1`          | DeepSeek V3.1            | 163K    |
| `openai/gpt-oss-120b`                | GPT-OSS 120B             | 131K    |
| `meta-llama/Llama-3.3-70B-Instruct`  | Llama 3.3 70B Instruct   | 131K    |
| `moonshotai/Kimi-K2.6`               | Kimi K2.6                | 262K    |

Use `GET /base-models?task_type=decoder&supports_inference=true` to see the current serverless catalog.

## Next steps

* [Synthetic Data](/guides/synthetic-data) — generate training data without manual annotation
* [Adaptive Inference](/guides/adaptive-inference) — automatically retrain on live production data
* [Agent Skills](/guides/agent-skills) — let an AI coding agent manage training and inference for you
* [Training Jobs API](/api-reference/training-jobs) — every endpoint, parameter, and response field