Skip to main content
Pioneer’s data generation API lets you produce high-quality labeled training examples without manually annotating data. You can generate synthetic examples from scratch for NER, classification, and decoder tasks, or bring your own unlabeled text and have Pioneer label it automatically. All generated data is saved directly to a named dataset ready for fine-tuning.
Generate endpoints are rate-limited to 120 requests per minute per user. For large datasets, consider batching your requests or using the job polling endpoint to monitor long-running generation jobs.

​
Start a generation job

POST /generate Starts an asynchronous job that generates labeled training examples and stores them in a named dataset. Returns a job ID you can use to poll for completion. Request body
​
string
required
The type of task to generate data for. Accepted values: ner, classification, decoder.
​
string
required
The name of the dataset to create or append to. If a dataset with this name already exists, new examples are added as a new version.
​
number
Number of labeled examples to generate.
​
string[]
List of label strings for NER or classification tasks. For NER, these are entity type names (e.g. "person", "organization"). For classification, these are the class names.
​
string
A natural-language description of the domain or topic for the generated examples. Providing a detailed description improves example quality and relevance.
​
object[]
Few-shot examples with labels to guide generation for classification tasks.
​
string
Custom instruction prompt to control generation style for decoder tasks.
Response
​
string
Unique identifier for the generation job. Use this with GET /generate/jobs/:job_id to poll for status.
​
string
Initial job status, typically queued.

​
Poll generation job status

GET /generate/jobs/:job_id Returns the current status of a data generation job. Poll this endpoint until the status is ready or failed before starting a training job on the resulting dataset. Path parameters
​
string
required
The job ID returned by POST /generate.
Response
​
string
The generation job ID.
​
string
Current job status. Values: queued, generating, ready,failed.
​
string
The dataset name that examples are being written to.
​
number
Number of examples generated so far.
​
string
The task type for this job (e.g. ner, classification).
​
string
Error message if the job failed, otherwise null.
​
string
ISO 8601 timestamp of when the job was created.

​
Auto-label text for NER

POST /generate/ner/label-existing Sends your own unlabeled text to Pioneer and returns NER annotations. Use this when you have existing text that you want to annotate rather than generating new synthetic examples. Request body
​
string[]
required
List of entity type names to detect. For example: ["person", "organization", "location"].
​
string[]
required
List of text strings to annotate. Accepts between 1 and 1,000 strings per request.
Response Returns an array of annotation objects, one per input string, each containing detected entities with their spans, labels, and confidence scores.

​
Auto-classify text

POST /generate/classification/label-existing Sends your own unlabeled text to Pioneer and returns classification labels. Use this when you have existing text that you want to classify rather than generating new synthetic examples. Request body
​
string[]
required
List of class names to classify text into. For example: ["positive", "negative", "neutral"].
​
string[]
required
List of text strings to classify. Accepts between 1 and 1,000 strings per request.
Response Returns an array of classification results, one per input string, each containing the predicted label and a confidence score.