Files
Jaaneek a0c031299b feat(web): add xAI Web Search provider plugin
Adds a new bundled web search provider plugin backed by xAI's agentic
Web Search tool (server-side `web_search` on the Responses API). Slots
in alongside the existing Firecrawl / Tavily / Exa / Brave / SearXNG /
DDGS providers; opt in via `web.backend: xai` (or auto-selected by the
registry's single-provider shortcut when it's the only available web
provider, matching every other backend's behavior).

Reuses the existing xAI HTTP credential plumbing (`tools/xai_http.py`)
so it works with both `hermes auth login xai-oauth` (SuperGrok OAuth)
and `XAI_API_KEY` — no new credential paths, no new env vars, no new
setup-wizard prompts. The existing `xai_grok` post_setup hook handles
credential collection.

Reference: https://docs.x.ai/developers/tools/web-search

Provider behavior
-----------------
- Sends a structured prompt to Grok with `tools=[{"type": "web_search"}]`
  enabled and `include=["no_inline_citations"]`, then parses results
  from a `{"results": [...]}` JSON block (primary), falling back to
  `url_citation` annotations (secondary) and the top-level `citations`
  list (last-ditch). Annotation fallback falls through to citations
  when no rows are extractable, so future annotation types xAI may
  add don't silently mask real data.
- HTTP 200 + `{"error": {...}}` envelopes (model-overload, refusal)
  are surfaced as failures rather than masked as success-with-empty-
  results.
- HTTP 401 on the OAuth path triggers a single `force_refresh=True`
  retry — closes two gaps the resolver's proactive JWT-exp shortcut
  doesn't cover: opaque (non-JWT) access tokens and mid-window
  revocation. Env-var (`XAI_API_KEY`) credentials never retry; they
  can't be refreshed and an immediate retry would just burn quota.
- `is_available()` is a cheap probe (env var OR auth.json read), never
  invokes the OAuth resolver — required by the ABC contract because
  it runs on every `hermes tools` repaint and at tool-registration time.
- Class docstring documents the LLM-in-a-trench-coat trust model so
  callers piping untrusted input into `web_search` know returned URLs
  are model-generated and should be validated before fetching.

Config (`config.yaml`):

    web:
      backend: xai
      xai:
        model: grok-4.3         # optional, defaults to grok-4.3
        allowed_domains:        # optional, max 5 — mutex with excluded_domains
          - arxiv.org
        excluded_domains:       # optional, max 5
          - example-spam.com
        timeout: 90             # optional, seconds

Files
-----
- plugins/web/xai/plugin.yaml          (new) plugin manifest
- plugins/web/xai/__init__.py          (new) register(ctx) hook
- plugins/web/xai/provider.py          (new) XAIWebSearchProvider impl
- tools/xai_http.py                    (+47) has_xai_credentials()
                                            cheap-probe helper +
                                            keyword-only force_refresh
                                            arg on resolve_xai_http_
                                            credentials() (backwards
                                            compatible; all 9 other
                                            call sites unaffected)
- tools/web_tools.py                   (+11) "xai" added to configured-
                                            backend set + branch in
                                            _is_backend_available()
- tests/tools/test_web_providers_xai.py (new, 39 tests) covers
                                        identity, cheap-probe semantics,
                                        JSON / annotation / citations
                                        parse paths, request payload
                                        shape, error envelopes, OAuth
                                        force-refresh-on-401 retry,
                                        env-var-no-retry guard, 500-not-
                                        retried guard, refresh-returns-
                                        same-token guard, OAuth runtime
                                        resolution, and backend wiring.

Tests
-----
- 39 xai-suite passes
- 79 sibling web-provider tests (brave-free, ddgs, searxng, base) pass
- 119 cross-suite tests for other xai_http callers (transcription,
  x_search, tts) pass — verifies the new keyword-only arg is BC
- scripts/check-windows-footguns.py: clean on all 5 modified files

No edits to run_agent.py, cli.py, gateway/, toolsets, config schema,
plugin core, or auth core.
2026-05-19 19:27:34 -07:00

129 lines
5.0 KiB
Python

"""Shared helpers for direct xAI HTTP integrations."""
from __future__ import annotations
import json
import os
from typing import Dict
def has_xai_credentials() -> bool:
"""Cheap probe — return True when xAI credentials are *likely* usable.
Deliberately avoids :func:`resolve_xai_http_credentials` so callers in
hot-paint paths (``hermes tools`` repaint, tool-registration scans,
``WebSearchProvider.is_available()``) don't incur disk locks or — in
the OAuth path — a network token refresh. The ABC contract on
:meth:`agent.web_search_provider.WebSearchProvider.is_available`
explicitly forbids network calls for exactly this reason.
Resolution order, fast-to-slow:
1. ``XAI_API_KEY`` env var (cheapest; covers explicit-key users).
2. ``~/.hermes/auth.json`` has a non-empty ``providers.xai-oauth.tokens.access_token``
(single file read, no expiry check, no refresh).
Returns False on any exception so a corrupted auth store can't block
other availability scans. Truthful refresh + expiry handling happens
in ``search()`` (or whichever caller actually makes the request).
"""
if os.environ.get("XAI_API_KEY", "").strip():
return True
try:
from hermes_constants import get_hermes_home
auth_path = get_hermes_home() / "auth.json"
if not auth_path.exists():
return False
store = json.loads(auth_path.read_text())
providers = store.get("providers") if isinstance(store, dict) else None
xai_state = providers.get("xai-oauth") if isinstance(providers, dict) else None
tokens = xai_state.get("tokens") if isinstance(xai_state, dict) else None
access_token = tokens.get("access_token") if isinstance(tokens, dict) else None
return bool(str(access_token or "").strip())
except Exception:
return False
def get_env_value(name: str, default=None):
"""Read ``name`` from ``~/.hermes/.env`` first, then ``os.environ``.
Wraps :func:`hermes_cli.config.get_env_value` so tests can patch
``tools.xai_http.get_env_value`` to inject dotenv-only secrets into the
xAI credential resolver.
"""
try:
from hermes_cli.config import get_env_value as _hermes_get_env_value
value = _hermes_get_env_value(name)
if value is not None:
return value
except Exception:
pass
return os.environ.get(name, default)
def hermes_xai_user_agent() -> str:
"""Return a stable Hermes-specific User-Agent for xAI HTTP calls."""
try:
from hermes_cli import __version__
except Exception:
__version__ = "unknown"
return f"Hermes-Agent/{__version__}"
def resolve_xai_http_credentials(*, force_refresh: bool = False) -> Dict[str, str]:
"""Resolve bearer credentials for direct xAI HTTP endpoints.
Prefers Hermes-managed xAI OAuth credentials when available, then falls back
to ``XAI_API_KEY`` resolved via ``hermes_cli.config.get_env_value`` so keys
stored in ``~/.hermes/.env`` (the standard Hermes location) are honored —
not just ones already exported into ``os.environ``. This keeps direct xAI
endpoints (images, TTS, STT, etc.) aligned with the main runtime auth model
and preserves the regression contract from PR #17140 / #17163.
Set ``force_refresh=True`` to bypass the resolver's JWT-exp shortcut and
perform an unconditional OAuth refresh. Callers should use this only as a
reactive remediation after a server 401 (mid-window revocation, opaque
tokens where the proactive JWT check is a no-op, etc.), not as a default —
the auth-store lock is held for the duration of the refresh.
"""
if not force_refresh:
try:
from hermes_cli.runtime_provider import resolve_runtime_provider
runtime = resolve_runtime_provider(requested="xai-oauth")
access_token = str(runtime.get("api_key") or "").strip()
base_url = str(runtime.get("base_url") or "").strip().rstrip("/")
if access_token:
return {
"provider": "xai-oauth",
"api_key": access_token,
"base_url": base_url or "https://api.x.ai/v1",
}
except Exception:
pass
try:
from hermes_cli.auth import resolve_xai_oauth_runtime_credentials
creds = resolve_xai_oauth_runtime_credentials(force_refresh=force_refresh)
access_token = str(creds.get("api_key") or "").strip()
base_url = str(creds.get("base_url") or "").strip().rstrip("/")
if access_token:
return {
"provider": "xai-oauth",
"api_key": access_token,
"base_url": base_url or "https://api.x.ai/v1",
}
except Exception:
pass
api_key = str(get_env_value("XAI_API_KEY") or "").strip()
base_url = str(get_env_value("XAI_BASE_URL") or "https://api.x.ai/v1").strip().rstrip("/")
return {
"provider": "xai",
"api_key": api_key,
"base_url": base_url,
}