AI导哈

智能体工作流

langsmith-online-eval-engineering

langchain-ai/la…n-skills

Iteratively inspect traces, interview the user, and create LangSmith online evaluators one at a time. Use specifically for creating online evaluators for use within LangSmith -- use "eval-engineering" for Harbor-style online evaluations.

  • 3346 次安装
  • 1300 星标
  • 收录于

安装命令

npx skills add https://github.com/langchain-ai/langchain-skills --skill langsmith-online-eval-engineering

SKILL.md

Online Eval Engineering

Build online evaluators iteratively:

inspect traces and interview user -> propose directions -> user chooses
-> build evaluator -> test, attach, verify -> review and repeat

Read references/langsmith-api.md before creating or modifying evaluators.

1. Inspect traces

Ask the user for their LangSmith project name. Fetch recent root-level traces and print their structure. Read references/trace-inspection.md. Find:

  • run name and type;
  • available input and output field names;
  • the shape and content of the data (truncated samples);
  • which fields carry the data an evaluator would need.

相关 Skills

deep-agents-memorylangchain-ai/la…n-skillsINVOKE THIS SKILL when your Deep Agent needs memory, persistence, or filesystem access. Covers StateBackend (ephemeral), StoreBackend (persistent), FilesystemMiddleware, and CompositeBackend for routing.智能体工作流1.9万langgraph-fundamentalslangchain-ai/la…n-skillsINVOKE THIS SKILL when writing ANY LangGraph code. Covers StateGraph, state schemas, nodes, edges, Command, Send, invoke, streaming, and error handling.营销与写作1.8万langgraph-persistencelangchain-ai/la…n-skillsINVOKE THIS SKILL when your LangGraph needs to persist state, remember conversations, travel through history, or configure subgraph checkpointer scoping. Covers checkpointers, thread_id, time travel, Store, and subgraph persistence modes.其他1.8万langchain-raglangchain-ai/la…n-skillsINVOKE THIS SKILL when building ANY retrieval-augmented generation (RAG) system. Covers document loaders, RecursiveCharacterTextSplitter, embeddings (OpenAI), and vector stores (Chroma, FAISS, Pinecone).其他1.8万langchain-fundamentalslangchain-ai/la…n-skillsCreate LangChain agents with create_agent, define tools, and use middleware for human-in-the-loop and error handling.其他1.8万langgraph-human-in-the-looplangchain-ai/la…n-skillsINVOKE THIS SKILL when implementing human-in-the-loop patterns, pausing for approval, or handling errors in LangGraph. Covers interrupt(), Command(resume=...), approval/validation workflows, and the 4-tier error handling strategy.智能体工作流1.7万