> ## Documentation Index
> Fetch the complete documentation index at: https://docs.corepillar.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluate a trace

> Runs every evaluator in a collection against the trace and stores the results. Offline (deterministic) evaluators run without a model. LLM-judge evaluators call the model in `model_config_id` with your organization's stored provider key, which is required if the collection contains any. Runs synchronously.

**Auth:** session. **Permission:** `run_agents` or `manage_agents`.



## OpenAPI

````yaml /api-reference/openapi.json post /projects/{projectId}/traces/{traceId}/evaluate
openapi: 3.1.0
info:
  title: CorePillar API
  version: 1.0.0
  description: >-
    REST API for AgentHub and ModelHub. Ingestion endpoints authenticate with a
    project tracing key; all other endpoints use the signed-in user's browser
    session.
servers:
  - url: https://pillarforge.corepillar.ai/api/v1
    description: AgentHub
security: []
tags:
  - name: Ingestion
    description: 'Send traces with a tracing key: JSON ingest and OpenTelemetry (OTLP).'
  - name: Traces
    description: Read traces and spans.
  - name: Evaluations
    description: Evaluate traces with evaluator collections.
  - name: Evaluators
    description: Built-in and custom evaluators.
  - name: Evaluator collections
    description: Ordered groups of evaluators.
  - name: Organizations
    description: Organizations you belong to.
  - name: Members
    description: Organization members, invites and roles.
  - name: Projects
    description: Projects inside an organization.
  - name: SSO
    description: Okta SSO configuration.
  - name: Tracing keys
    description: Create, list and revoke project tracing keys.
  - name: Cost calculator
    description: Estimate, generate and execute scenario runs.
  - name: Synthetic
    description: Synthetic agent and data setup from source material.
  - name: Chat
    description: ModelHub chat. Served at https://modelhub.corepillar.ai/api/v1.
  - name: Auth
    description: Sign-in helpers.
  - name: E2E
    description: Internal end-to-end encryption key directory.
paths:
  /projects/{projectId}/traces/{traceId}/evaluate:
    post:
      tags:
        - Evaluations
      summary: Evaluate a trace
      description: >-
        Runs every evaluator in a collection against the trace and stores the
        results. Offline (deterministic) evaluators run without a model.
        LLM-judge evaluators call the model in `model_config_id` with your
        organization's stored provider key, which is required if the collection
        contains any. Runs synchronously.


        **Auth:** session. **Permission:** `run_agents` or `manage_agents`.
      operationId: evaluateTrace
      parameters:
        - name: projectId
          in: path
          required: true
          description: Project id.
          schema:
            type: string
            format: uuid
        - name: traceId
          in: path
          required: true
          description: Your `trace_id`, URL-encoded.
          schema:
            type: string
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - collection_id
              properties:
                collection_id:
                  type: string
                  format: uuid
                model_config_id:
                  type:
                    - string
                    - 'null'
                  format: uuid
                  description: >-
                    Judge model from Model Manager. Required when the collection
                    has LLM-judge evaluators.
      responses:
        '201':
          description: Run completed.
          content:
            application/json:
              schema:
                type: object
                required:
                  - data
                properties:
                  data:
                    type: object
                    properties:
                      runId:
                        type: string
                        format: uuid
                      averageScore:
                        type: number
                      passRate:
                        type: number
                      results:
                        type: array
                        items:
                          $ref: '#/components/schemas/TraceEvalResult'
        '400':
          description: Validation failed. `error.code` is `VALIDATION_ERROR`.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorEnvelope'
        '401':
          description: Not signed in (`UNAUTHORIZED`).
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorEnvelope'
        '403':
          description: >-
            Not a member of the organization, or missing the required permission
            (`FORBIDDEN`).
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorEnvelope'
        '404':
          description: >-
            Project not found (`PROJECT_NOT_FOUND`) or trace not found
            (`NOT_FOUND`).
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorEnvelope'
        '500':
          description: >-
            Unexpected error, including a missing or unknown judge model
            (`INTERNAL_ERROR`).
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ErrorEnvelope'
      security:
        - session: []
components:
  schemas:
    TraceEvalResult:
      type: object
      properties:
        id:
          type: string
          format: uuid
        run_id:
          type: string
          format: uuid
        evaluator_id:
          type: string
          format: uuid
        evaluator_name:
          type: string
        eval_type:
          type: string
          enum:
            - offline_deterministic
            - online_llm_judge
        score:
          type: number
          minimum: 0
          maximum: 1
        passed:
          type: boolean
        reasoning:
          type: string
        created_at:
          type: string
          format: date-time
    ErrorEnvelope:
      type: object
      required:
        - error
      properties:
        error:
          type: object
          required:
            - code
            - message
          properties:
            code:
              type: string
              description: Stable machine-readable code. Branch on this, not on `message`.
              examples:
                - VALIDATION_ERROR
            message:
              type: string
              description: Human-readable description. May change.
            details:
              type: object
              additionalProperties: true
              description: Extra context; often empty.
  securitySchemes:
    session:
      type: apiKey
      in: cookie
      name: session
      description: >-
        The session cookie set when you sign in to AgentHub or ModelHub in a
        browser. The cookie name shown here is illustrative. There are no API
        tokens for these endpoints.

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.