AI导哈

测试与安全

arize-instrumentation-health

arize-ai/arize-skills

Audits instrumentation health of existing Arize traces. Runs deterministic checks over a bounded span sample (orphaned/uncategorized/duplicate spans, flat structure, blank root I/O, unset status, missing token counts or children) and returns a ranked report. Use when the user asks why traces look empty/flat/broken, wants to verify instrumentation is healthy, find instrumentation issues, or why evals or token/cost dashboards show n/a or zero. To debug app behavior or errors, use arize-trace.

  • 1046 次安装
  • 67 星标
  • 收录于

安装命令

npx skills add https://github.com/arize-ai/arize-skills --skill arize-instrumentation-health

SKILL.md

Arize Instrumentation Health Skill

Use this skill for an on-demand instrumentation health audit over a project's existing traces — the aggregate counterpart to arize-instrumentation (which verifies a single new trace) and arize-trace (which exports and inspects spans). It answers questions like:

  • "Why do my traces look empty or flat?"
  • "Check whether my Arize instrumentation is healthy."
  • "Find instrumentation issues in this project."
  • "Why are my evals / token / cost dashboards showing n/a or zero?"

Workflow

  1. Resolve scope — get the project (and space, if needed). If ambiguous, ask; do not guess.
  2. Export a bounded span sample using the arize-trace skill — do not hand-roll ax flags here. Follow its export guidance: start with a small sample scoped by --start-time to a recent window, into --output-dir .arize-tmp-traces. Pull ~20 traces' worth of spans for a full audit (see minimum-data rules below).
  3. Group spans by trace (context.trace_id); within each trace identify the root (parent_id/parent_span_id is null).
  4. Run the deterministic checks in references/checks.md against the sample.
  5. Report findings ranked by severity then confidence, using the Output format in references/checks.md.

This skill is read-only by default. Inspect exported spans and source files only when they help attribute the cause. Do not edit application code, tests, configuration, dependencies, or generated artifacts during a health audit unless the user explicitly asks this skill to make fixes in the same turn. When fixes are needed and the user has not asked for them in this turn, report the next action as a handoff to arize-instrumentation or the relevant framework-specific instrumentation path.

相关 Skills

arize-instrumentationarize-ai/arize-skillsAdds Arize AX tracing to an LLM application for the first time. Detects the stack, routes to the single matching integration doc, wires auto-instrumentation after user confirmation, and verifies traces land. Use when the user wants to instrument their app, add tracing from scratch, set up LLM observability, integrate OpenTelemetry or openinference, or get started with Arize tracing.数据与研究3492arize-prompt-optimizationarize-ai/arize-skillsOptimizes, improves, and debugs LLM prompts using production trace data, evaluations, and annotations. Extracts prompts from spans, gathers performance signal, and runs a data-driven optimization loop using the ax CLI. Use when the user mentions optimize prompt, improve prompt, make AI respond better, improve output quality, prompt engineering, prompt tuning, or system prompt improvement.智能体工作流3331arize-tracearize-ai/arize-skillsDownloads, exports, and inspects existing Arize traces and spans to understand what an LLM app is doing or debug runtime issues. Covers exporting traces by ID, spans by ID, sessions by ID, and root-cause investigation using the ax CLI. Use when the user wants to look at existing trace data, see what their LLM app is doing, export traces, download spans, investigate errors, or analyze behavior regressions.数据与研究3271arize-experimentarize-ai/arize-skillsCreates, runs, and analyzes Arize experiments for evaluating and comparing model performance. Covers experiment CRUD, exporting runs, comparing results, and evaluation workflows using the ax CLI. Use when the user mentions create experiment, run experiment, compare models, model performance, evaluate AI, experiment results, benchmark, A/B test models, or measure accuracy.智能体工作流3158arize-datasetarize-ai/arize-skillsCreates, manages, and queries Arize datasets and examples. Covers dataset CRUD, appending examples, exporting data, and file-based dataset creation using the ax CLI. Use when the user needs test data, evaluation examples, or mentions create dataset, list datasets, export dataset, append examples, dataset version, golden dataset, or test set.测试与安全3155arize-evaluatorarize-ai/arize-skillsHandles LLM-as-judge and code evaluator workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions create evaluator, LLM judge, code evaluator, hallucination, faithfulness, correctness, relevance, run eval, score spans, score experiment, trigger-run, column mapping, continuous monitoring, or improve evaluator prompt.智能体工作流3127