使用说明

RHOAI 3.4上的OGX（前身为LlamaStack）

使用LlamaStackDistribution自定义资源在OpenShift AI 3.4上部署具有MCP服务器集成的OGX服务器。

迁移说明： OGX v0.7.1附带RHOAI 3.4，取代了RHOAI 3.3中的LlamaStack v0.4.x。看从RHOAI 3.3迁移打破变化。

建筑

External vLLM Endpoints
  (Inference + Embedding)
         |
         v
+---------------------+         +-------------------------+
| OGX Server          |--SSE--->| OpenShift MCP Server    |
| (port 8321)         |         | (port 8080)             |
|                     |         +-------------------------+
| Providers:          |
|  - vllm-inference   |         +-------------------------+
|    (remote::vllm)   |--SSE--->| VictoriaMetrics MCP     |
|  - vllm-embedding   |         | Aggregator (port 8000)  |
|    (remote::vllm)   |         +-------------------------+
|  - inline::builtin  |
|    (responses)      |         +-------------------------+
|  - model-context-   |--SSE--->| Incident Detection MCP  |
|    protocol         |         | (port 8085)             |
|                     |         +-------------------------+
| APIs:               |
|  - inference         |
|  - responses         |
|  - tool_runtime      |
|  - vector_io         |
|  - files             |
+---------------------+
         |
         v
+---------------------+
| PostgreSQL          |
| (KV + SQL storage)  |
+---------------------+

先决条件

启用LlamaStack/OGX操作员的OpenShift AI 3.4 (llamastackoperator.managementState: Managed 在您的DataSciencesCluster CR中）
oc CLI已登录到集群
envsubst 可用（部分 gettext)
外部vLLM推理端点URL（以结尾 /v1)和API令牌
外部vLLM嵌入端点URL和API令牌

快速开始

# 1. Clone the repo
git clone https://github.com/ravisharma5/rhoai-llamastack.git
cd rhoai-llamastack

# 2. Configure your environment
cp .env.example .env
# Edit .env with your vLLM endpoints and API tokens

# 3. Login to your OpenShift cluster
oc login --server=https://api.your-cluster.com:6443

# 4. Deploy everything
./deploy.sh

# 5. Verify
oc get pods
OGX_URL=https://$(oc get route llama-stack-server -o jsonpath='{.spec.host}')
curl -sk $OGX_URL/v1/health
curl -sk $OGX_URL/v1/models

逐步部署

1.创建命名空间

oc new-project llama-stack

2.配置和创建机密

cp .env.example .env
# Edit .env with your values:
#   VLLM_URL, VLLM_API_TOKEN
#   VLLM_EMBEDDING_URL, VLLM_EMBEDDING_API_TOKEN
#   POSTGRES_PASSWORD
source .env
envsubst < llama-stack-secret.yaml | oc apply -f -

3.创建OGX服务器配置

该配置使用OGX v0.7.x模式。与v0.4.x的主要区别：

version: 2
distro_name: rh                    # was: image_name

apis:
  - inference
  - responses                      # was: agents
  - tool_runtime
  - vector_io
  - files
  # Removed: eval, scoring, datasetio, safety

providers:
  inference:
    - provider_id: vllm-inference
      provider_type: remote::vllm
      config:
        base_url: ${env.VLLM_URL}  # must be base_url, not url
        max_tokens: 8192
        api_token: ${env.VLLM_API_TOKEN}
        network:                   # was: tls_verify (flat field)
          tls:
            verify: false

  responses:                       # was: agents
    - provider_id: builtin         # was: meta-reference
      provider_type: inline::builtin  # was: inline::meta-reference
      config:
        persistence:
          responses:
            backend: sql_default
            table_name: agents_responses

  tool_runtime:
    - provider_id: model-context-protocol
      provider_type: remote::model-context-protocol
      config: {}

应用ConfigMap：

oc create configmap llama-stack-run-config \
  --from-file=config.yaml=llamastack-config.yaml

重要提示： ConfigMap密钥必须是 config.yaml。操作员将其安装在 /etc/llama-stack/config.yaml.

4.部署PostgreSQL

oc apply -f postgres.yaml
oc wait --for=condition=available deployment/postgres-llamastack --timeout=120s

5.部署OpenShift MCP服务器

oc apply -f openshift-mcp-server.yaml
oc wait --for=condition=available deployment/openshift-mcp-server --timeout=120s

6.部署OGX服务器（LlamaStackDistribution CR）

CR命令RHOAI操作员部署和管理OGX服务器：

apiVersion: llamastack.rhoai.opendatahub.io/v1alpha1
kind: LlamaStackDistribution
metadata:
  name: llama-stack-server
spec:
  replicas: 1
  userConfig:                      # must be userConfig, not userConfigMapRef
    configMapName: llama-stack-run-config
  secretRef:
    name: llama-stack-secret
  network:
    allowedFrom:
      namespaces: ["*"]            # allows OpenShift router traffic

oc apply -f llamastackdistribution.yaml

7.创建外部路由

oc create route edge llama-stack-server \
  --service=llama-stack-server-service --port=8321

8.验证部署

# Check pods
oc get pods

# Follow logs
oc logs -l app.kubernetes.io/instance=llama-stack-server -f

# Health check
OGX_URL=https://$(oc get route llama-stack-server -o jsonpath='{.spec.host}')
curl -sk $OGX_URL/v1/health

# List models
curl -sk $OGX_URL/v1/models

# List registered toolgroups (still works in v0.7.x)
curl -sk $OGX_URL/v1/toolgroups

响应API

这 /v1/responses API处理服务器端的完整代理循环：工具发现、LLM驱动的工具选择、执行和响应生成。

基本查询

from llama_stack_client import LlamaStackClient

client = LlamaStackClient(base_url="https://<ogx-route-url>")

response = client.responses.create(
    model="vllm-inference/your-model-id",
    input="List all pods in the llama-stack namespace",
    tools=[
        {
            "type": "mcp",
            "server_label": "openshift",
            "server_url": "http://openshift-mcp-server:8080/sse",
            "require_approval": "never",
        }
    ],
    stream=False,
)

v0.7.x中的新参数

response = client.responses.create(
    model="vllm-inference/your-model-id",
    input="Investigate high CPU usage across all namespaces",
    tools=mcp_tools,
    stream=True,
    instructions="You are an AIOps assistant...",
    # New in v0.7.x:
    parallel_tool_calls=True,       # concurrent MCP queries
    max_output_tokens=4096,         # per-request output limit
    max_tool_calls=10,              # total tool call cap
    reasoning={"effort": "medium"}, # reasoning depth control
    truncation="auto",              # was: extra_body={"truncation": "auto"}
)

参数	用途	默认值
`parallel_tool_calls`	允许模型每转发出多个工具调用	`True`
`max_output_tokens`	每请求输出令牌限制	模型默认值
`max_tool_calls`	限制每个请求的工具调用总数	无限制
`reasoning`	控制推理深度(`none`/`minimal`/`low`/`medium`/`high`/`xhigh`)	无
`truncation`	上下文窗口管理策略	无

流媒体活动

for event in response:
    if event.type == "response.output_text.delta":
        print(event.delta, end="")
    elif event.type == "response.output_item.added":
        if event.item.type == "mcp_call":
            print(f"Calling: {event.item.name}")
    elif event.type == "response.output_item.done":
        if event.item.type == "mcp_call":
            print(f"Result: {event.item.output[:200]}")
    elif event.type == "response.completed":
        print("Done")
    elif event.type == "response.failed":
        print(f"Error: {event.response.error}")

MCP服务器配置

注册MCP工具组

MCP服务器已在OGX配置中注册 registered_resources.tool_groups:

registered_resources:
  tool_groups:
    - toolgroup_id: mcp::openshift
      provider_id: model-context-protocol
      mcp_endpoint:
        uri: http://openshift-mcp-server:8080/sse

    - toolgroup_id: mcp::victoriametrics
      provider_id: model-context-protocol
      mcp_endpoint:
        uri: http://victoriametrics-mcp-aggregator:8000/sse

安全模式

模式	标志	它允许什么
只读	`--read-only`	仅列出、获取、描述
非破坏性	`--disable-destructive`	读取+创建（无删除/更新）
完全访问	（无标志）	所有操作

身份验证（OAuth2/密钥斗篷）

OGX支持通过Keycloak进行OAuth2令牌身份验证：

server:
  port: 8321
  auth:
    provider_config:
      type: "oauth2_token"
      jwks:
        uri: "http://keycloak-service.rhbk.svc:8080/realms/aiops/protocol/openid-connect/certs"
      issuer: "https://keycloak-route/realms/aiops"
      audience: "account"

使用auth的客户端使用情况：

client = LlamaStackClient(
    base_url="https://<ogx-route-url>",
    api_key="<jwt-token>",
)

从RHOAI 3.3迁移

突破性变化（v0.4.x→ v0.7.x）

区域	旧（v0.4.x）	新（v0.7.x）
配置字段	`image_name: rh`	`distro_name: rh`
代理商提供商	`inline::meta-reference`	`inline::builtin`
代理商API	`agents` 在api列表中	`responses` 在api列表中
TLS配置	`tls_verify: false`	`network: { tls: { verify: false }}`
RAG工具组	`builtin::rag`	`builtin::file_search`
SDK版本	`llama-stack-client>=0.4.0,<0.5`	`llama-stack-client>=0.7.0,<0.8`
已删除API	--	`eval`, `scoring`, `datasetio`, `safety`
已删除部分	--	`registered_resources.tool_groups` 自动寄存器

SDK更改

# Import path is UNCHANGED
from llama_stack_client import LlamaStackClient

# New parameters available on responses.create():
# parallel_tool_calls, max_output_tokens, max_tool_calls, reasoning, truncation

# max_infer_iters replaced by max_tool_calls
# extra_body={"truncation": "auto"} replaced by truncation="auto"

添加远程OpenShift群集

请参阅本README先前版本中的远程集群设置指南。过程不变——部署第二个MCP服务器pod，其中kubeconfig指向远程集群，将其注册为 mcp::openshift-remote 在OGX配置中。

更新配置

oc create configmap llama-stack-run-config \
  --from-file=config.yaml=llamastack-config.yaml \
  --dry-run=client -o yaml | oc apply -f -

# Restart to pick up changes
oc rollout restart deployment/llama-stack-server

拆除

./teardown.sh
# Or manually:
oc delete llamastackdistribution llama-stack-server
oc delete route llama-stack-server
oc delete configmap llama-stack-run-config
oc delete -f openshift-mcp-server.yaml
oc delete -f postgres.yaml
envsubst < llama-stack-secret.yaml | oc delete -f -

故障排除

CrashLoopBackOff中的OGX吊舱：

检查日志： oc logs -l app.kubernetes.io/instance=llama-stack-server
验证ConfigMap密钥是否 config.yaml （不是 run.yaml)
验证 base_url （不是 url)在vLLM提供程序配置中
验证 distro_name （不是 image_name)
验证 inline::builtin （不是 inline::meta-reference)

路线返回503：

检查网络策略： oc get networkpolicy -o yaml
确保 network.allowedFrom.namespaces: ["*"] 在CR

MCP工具未出现：

验证配置中的工具组条目 registered_resources.tool_groups
验证 model-context-protocol 提供者在 providers.tool_runtime
检查OGX日志中的MCP连接错误

身份验证错误（401/403）：

验证是否可以从OGX pod访问Keycloak JWKS URI
验证 issuer 匹配外部Keycloak路由URL（JWT iss 声明使用外部URL）
检查令牌到期时间： python3 -c "import jwt; print(jwt.decode(token, options={'verify_signature':False}))"

升级后的无声精度回归：

以原子方式升级OGX和vLLM——红帽基准测试显示，当只升级一个时，准确性会下降
前后运行测试查询以验证质量