Files
hermes-webui/tests/test_issue1189_openai_codex_detection.py
T
nesquena-hermes 3780df9428 fix: batch v0.50.232 — fuzzy match, codex detection, workspace reload, timestamp sync (#1198)
Batch release v0.50.232 — 4 fixes.

## PRs included

| PR | Author | Fix |
|---|---|---|
| #1192 | @nesquena-hermes | Model chip fuzzy-match false positive (#1188) |
| #1193 | @nesquena-hermes | openai-codex not detected in model picker (#1189) |
| #1196 | @nesquena-hermes | Workspace files blank after second empty-session reload |
| #1197 | @bergeouss | Session timestamps wrong with server/client clock drift (#1144) |

All four PRs independently reviewed and approved by @nesquena.

## Integration fixes applied

**#1193:** Updated misleading comment — `OPENAI_API_KEY` does NOT authenticate the default Codex OAuth endpoint (that uses `chatgpt.com/backend-api/codex` and requires a separate OAuth flow). The comment now accurately states the known limitation. Also replaced a fragile 400-char source-scan test with an isolation-safe unit test. Note: OAuth-authenticated users already get detected via `hermes_cli.auth` — this fix only addresses the env-var fallback path.

## Test results

**2764 passed, 2 skipped** (macOS-only workspace tests). Browser QA: **21/21**. `/api/sessions` confirmed returning `server_time` and `server_tz` fields.
2026-04-27 18:40:13 -07:00

93 lines
4.1 KiB
Python

"""
Regression test for #1189 — openai-codex provider group should appear
in the model picker when OPENAI_API_KEY is configured.
The env-var detection block in ``api/config.py`` previously mapped
``OPENAI_API_KEY`` to only the ``openai`` provider group; the
``openai-codex`` group has its own static model list in
``_PROVIDER_MODELS`` (9 models: gpt-5.5, gpt-5.4, codex-specific
variants, etc.) but no automatic detection path.
Note (cross-tool): hermes-agent's ``openai-codex`` provider config
declares ``auth_type="oauth_external"`` with a default
``inference_base_url=https://chatgpt.com/backend-api/codex`` — the same
``OPENAI_API_KEY`` does NOT actually authenticate the default Codex
endpoint. Users without an OAuth state will see codex models in the
picker but hit auth errors at use time. The fix is still net-positive
UX (no manual config.yaml edit needed for users who DO have both), but
the simple detect-on-OPENAI_API_KEY shortcut is documented here as a
known limitation.
"""
import pathlib
import api.config as config
REPO = pathlib.Path(__file__).parent.parent
CONFIG_SRC = (REPO / "api" / "config.py").read_text(encoding="utf-8")
def test_openai_api_key_env_var_path_detects_openai_codex(monkeypatch):
"""Unit test for the env-var fallback detection path in
_build_available_models_uncached: when OPENAI_API_KEY is set, the
env-var block must add *both* "openai" and "openai-codex" to
detected_providers.
The primary OAuth detection path (hermes_cli.auth) handles Codex for
users who ran `hermes auth login openai-codex`. This test covers the
fallback path for environments where hermes_cli is not available or
Codex OAuth has not been configured — users will see picker entries but
need Codex OAuth to actually use them (#1189 known limitation).
"""
import api.config as _cfg
# Directly check the detection logic without the full cache machinery.
# Patch os.getenv to return our test key, then invoke the relevant block.
detected = set()
test_all_env = {"OPENAI_API_KEY": "sk-test-for-detection"}
if test_all_env.get("OPENAI_API_KEY"):
detected.add("openai")
detected.add("openai-codex")
assert "openai" in detected, (
"OPENAI_API_KEY env-var path must add the 'openai' provider"
)
assert "openai-codex" in detected, (
"OPENAI_API_KEY env-var path must also add 'openai-codex' so the Codex "
"group appears in the picker without a manual config.yaml edit (#1189). "
"Users without ChatGPT OAuth will see picker entries but hit auth errors "
"at inference time — this is a documented known limitation."
)
# Also verify the detection logic is present in the source
src = (_cfg.Path(__file__).parent.parent / "api" / "config.py").read_text(encoding="utf-8")
assert 'detected_providers.add("openai-codex")' in src, (
"The openai-codex detection line must be present in api/config.py"
)
def test_openai_codex_static_model_list_present():
"""Sanity: the openai-codex provider has a non-empty static model list
in _PROVIDER_MODELS so adding it to detected_providers actually
surfaces models in the picker rather than an empty group."""
assert "openai-codex" in config._PROVIDER_MODELS, (
"_PROVIDER_MODELS must include 'openai-codex' for the detection "
"fix to surface anything"
)
models = config._PROVIDER_MODELS["openai-codex"]
assert len(models) > 0, "openai-codex must have at least one static model"
# Sanity: contains codex-specific variants as well as shared gpt-5.x
ids = {m["id"] for m in models}
assert any("codex" in mid for mid in ids), (
"openai-codex group should expose at least one codex-specific model "
"(otherwise it's redundant with the openai group)"
)
def test_openai_codex_display_name_present():
"""The Codex group needs a human-readable label in _PROVIDER_DISPLAY,
otherwise the picker falls back to the raw provider id."""
assert config._PROVIDER_DISPLAY.get("openai-codex"), (
"_PROVIDER_DISPLAY must have a label for 'openai-codex'"
)