> ## Documentation Index > Fetch the complete documentation index at: https://docs.corepillar.ai/llms.txt > Use this file to discover all available pages before exploring further. # Tracing quickstart > Create a tracing key, send your first trace over HTTP, and find it in Traces. AgentHub stores what your agent did as **traces**. A trace is one run of your agent (one request, one conversation turn, one job). It holds a list of **spans**, one per step: a model call, a tool call, an HTTP request, a database query, or any function you want to time. You send traces to AgentHub with a plain HTTPS `POST`. There is no CorePillar SDK package to install. The snippets on this page are ordinary HTTP calls you can copy into any language. If your agent already emits OpenTelemetry spans (for example through OpenLLMetry, OpenInference or the standard OTel SDKs), you can skip the JSON format entirely and point your exporter at AgentHub. See [OpenTelemetry (OTLP)](/agenthub/developers/opentelemetry). ## What you need * An AgentHub project. Every trace belongs to exactly one project, chosen by the key you send it with. * A tracing key for that project. Creating one needs the `manage_org_settings` permission, which the **Owner** and **Admin** roles have. See [Roles and permissions](/agenthub/settings/roles-and-permissions). * Anything that can make an HTTPS request: `curl`, Python, Node.js, or your agent's own HTTP client. ## Steps In AgentHub, open **Settings** and choose **Tracing keys**. 1. Pick the **Project** the traces should land in. 2. Pick an **Environment**: **Live** creates a `tk_live_` key, **Test** creates a `tk_test_` key. Both work the same way for ingestion; the prefix just helps you keep production and test credentials apart. 3. Click **+ New live key** (or **+ New test key**). 4. Copy the key from the banner with **Copy key**. The full key is shown once. AgentHub stores only a SHA-256 hash of it plus a short hint, so it cannot show you the key again. If you lose it, create a new one and revoke the old one. Keys look like `tk_live_` or `tk_test_` followed by 48 lowercase hex characters. Full details, including revoking, are in [Tracing keys](/agenthub/settings/tracing-keys). Treat the key like a password. Put it in an environment variable or your secret manager, never in source control. ```bash theme={null} export COREPILLAR_TRACING_KEY="tk_live_your_key_here" ``` The variable name is up to you. The examples below read `COREPILLAR_TRACING_KEY`. `POST` a JSON trace to the ingest endpoint: ```text theme={null} POST https://pillarforge.corepillar.ai/api/v1/traces/ingest ``` Pass the key as a Bearer token. The example sends one trace with two spans: a root `agent_run` span and a child model call. ```bash curl theme={null} curl -X POST https://pillarforge.corepillar.ai/api/v1/traces/ingest \ -H "Authorization: Bearer $COREPILLAR_TRACING_KEY" \ -H "Content-Type: application/json" \ -H "Idempotency-Key: quickstart-0001" \ -d '{ "trace_id": "quickstart-0001", "root_name": "agent_run", "environment": "development", "tags": ["quickstart"], "spans": [ { "span_id": "root", "name": "agent_run", "kind": "function", "input": { "user": "What is the capital of France?" }, "output": { "content": "Paris." }, "started_at": "2026-03-17T10:00:00.000Z", "ended_at": "2026-03-17T10:00:01.200Z" }, { "span_id": "llm-1", "parent_span_id": "root", "name": "chat.completions.create", "kind": "llm", "model": "gpt-4o-mini", "tokens_in": 120, "tokens_out": 45, "cost_usd": 0.00012, "latency_ms": 890, "input": { "user": "What is the capital of France?" }, "output": { "content": "Paris." }, "started_at": "2026-03-17T10:00:00.100Z", "ended_at": "2026-03-17T10:00:00.990Z" } ] }' ``` ```python Python theme={null} import os import uuid from datetime import datetime, timedelta, timezone import requests INGEST_URL = "https://pillarforge.corepillar.ai/api/v1/traces/ingest" TRACING_KEY = os.environ["COREPILLAR_TRACING_KEY"] def iso(dt: datetime) -> str: # AgentHub needs UTC with a trailing "Z". datetime.isoformat() writes # "+00:00", which the API rejects. return dt.astimezone(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z" start = datetime.now(timezone.utc) trace_id = f"run-{uuid.uuid4()}" trace = { "trace_id": trace_id, "root_name": "agent_run", "environment": "development", "tags": ["quickstart"], "spans": [ { "span_id": "root", "name": "agent_run", "kind": "function", "input": {"user": "What is the capital of France?"}, "output": {"content": "Paris."}, "started_at": iso(start), "ended_at": iso(start + timedelta(milliseconds=1200)), }, { "span_id": "llm-1", "parent_span_id": "root", "name": "chat.completions.create", "kind": "llm", "model": "gpt-4o-mini", "tokens_in": 120, "tokens_out": 45, "cost_usd": 0.00012, "latency_ms": 890, "input": {"user": "What is the capital of France?"}, "output": {"content": "Paris."}, "started_at": iso(start + timedelta(milliseconds=100)), "ended_at": iso(start + timedelta(milliseconds=990)), }, ], } response = requests.post( INGEST_URL, json=trace, headers={ "Authorization": f"Bearer {TRACING_KEY}", # Reuse the same key if you retry this exact trace. "Idempotency-Key": trace_id, }, timeout=10, ) response.raise_for_status() print(response.status_code, response.json()) ``` ```typescript TypeScript theme={null} const INGEST_URL = "https://pillarforge.corepillar.ai/api/v1/traces/ingest"; const tracingKey = process.env.COREPILLAR_TRACING_KEY!; const start = Date.now(); const iso = (offsetMs: number) => new Date(start + offsetMs).toISOString(); // UTC, ends in "Z" const traceId = `run-${crypto.randomUUID()}`; const trace = { trace_id: traceId, root_name: "agent_run", environment: "development", tags: ["quickstart"], spans: [ { span_id: "root", name: "agent_run", kind: "function", input: { user: "What is the capital of France?" }, output: { content: "Paris." }, started_at: iso(0), ended_at: iso(1200), }, { span_id: "llm-1", parent_span_id: "root", name: "chat.completions.create", kind: "llm", model: "gpt-4o-mini", tokens_in: 120, tokens_out: 45, cost_usd: 0.00012, latency_ms: 890, input: { user: "What is the capital of France?" }, output: { content: "Paris." }, started_at: iso(100), ended_at: iso(990), }, ], }; const response = await fetch(INGEST_URL, { method: "POST", headers: { Authorization: `Bearer ${tracingKey}`, "Content-Type": "application/json", // Reuse the same key if you retry this exact trace. "Idempotency-Key": traceId, }, body: JSON.stringify(trace), }); if (response.status !== 202) { throw new Error(`Ingest failed: ${response.status} ${await response.text()}`); } console.log(await response.json()); ``` The TypeScript example runs on Node.js 18 or later (global `fetch` and `crypto.randomUUID`), Deno, Bun, or in an edge runtime. Don't call the ingest endpoint from a browser: that would expose your tracing key to anyone who opens developer tools. A successful ingest returns **`202 Accepted`**: ```json theme={null} { "data": { "trace_id": "quickstart-0001", "project_id": "6f1c2a9e-3b0d-4c55-9a51-2f7d8e4b1c3a", "span_count": 2, "status": "OK" } } ``` `span_count` and `status` describe the batch you just sent. The trace is stored by the time you get this response, so you can open it straight away. If something is wrong you get an error envelope instead, for example `401 UNAUTHORIZED` for a bad key or `400 VALIDATION_ERROR` for a malformed payload. See [Errors](/api-reference/errors). In AgentHub, open the project the key belongs to and click **Traces** in the sidebar. Your trace appears at the top of the list, newest first, under its `root_name`. You can also search by `trace_id`. Open it to see each span with its kind, model, tokens, cost, latency, input and output. The [Traces guide](/agenthub/traces) walks through the trace view. ## Common first-trace problems | Symptom | Likely cause | Fix | | - | - | - | | `401` "Tracing key required." | No key was sent, or the header was `authorization: bearer ...` with a lowercase `bearer`. | Send `Authorization: Bearer ` with a capital B, or use the `x-tracing-key` header. | | `401` "Invalid tracing key format." | The key was truncated, has a trailing character, or was pasted with a Unicode ellipsis (`…`) from the masked hint. | Copy the full key again. It must match `tk_live_` or `tk_test_` plus 48 hex characters. | | `401` "Invalid or revoked tracing key." | The key was revoked, or it was never created in this AgentHub. | Create a new key under **Settings → Tracing keys**. | | `400` with `fieldErrors.spans: ["Invalid datetime"]` | A timestamp has a timezone offset like `+00:00`, a space instead of `T`, or no time part. | Send UTC ISO 8601 ending in `Z`, for example `2026-03-17T10:00:00.000Z`. | | `400` "Request body must be valid JSON." | The body isn't JSON (often a shell quoting problem with `curl -d`). | Validate the body with a JSON linter, or send it from a file with `-d @trace.json`. | | Trace is in the wrong project | Each key belongs to one project. | Use the key created for the project you're looking at. | ## Next steps Every field, type, default and limit in the ingest payload. Retry safely, send a trace in several batches, and handle errors. Send spans from an existing OpenTelemetry setup with no code changes. The full API reference, including authentication and errors. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.