From a2b793be4fc7cb5fa1975f11ce8d24a92633032b Mon Sep 17 00:00:00 2001 From: nesquena-hermes Date: Sun, 3 May 2026 21:44:22 +0000 Subject: [PATCH 1/6] fix(picker): Nous Portal featured-set cap + endpoint symmetry (closes #1567) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two related dropdown bugs in one PR — same root shape (model-picker endpoints disagreeing about which Nous Portal models exist) plus the preemptive UX guard against the picker becoming unusable on large-tier Nous accounts. #1567 — Endpoint disagreement ============================= Reporter (Deor, Discord, May 03 2026) saw Settings → Providers card showing "Nous Portal — 396 models · OAuth" while the in-conversation picker dropdown listed only the four hardcoded curated entries. Two structural causes: 1. ``api/providers.py:get_providers`` iterates ALL OAuth providers regardless of authentication state and unconditionally live-fetches the catalog. 2. ``api/config.py:_build_available_models_uncached`` only iterates providers in ``detected_providers``, gated on ``hermes_cli.models.list_available_providers().authenticated``. That flag can disagree with ``get_auth_status().logged_in`` on some hermes_cli versions. When the disagreement happens for Nous, the picker silently falls through to the curated 4-entry static list while the providers card keeps showing the live catalog — exactly the asymmetry users report. Plus: the Nous live-fetch branch in `_build_available_models_uncached` fell back to the same curated 4-entry list when `provider_model_ids` returned an empty list (transient failure / OAuth refresh in flight), which doubles down on the disagreement instead of healing it. UX cap (the design concern Nathan flagged on triage) ==================================================== Even with the disagreement fixed, dumping a 397-model catalog into a flat dropdown is unusable. We trim the visible picker to a curated ~15-entry featured set when the catalog exceeds 25 models, and surface the rest under a new ``extra_models`` field so: - ``/model`` slash autocomplete (commands.js) covers the full catalog - ``_dynamicModelLabels`` (ui.js) hydrates from both lists, so a model selected from outside the featured slice still gets a proper label - The optgroup label gets ``" (15 of 397)"`` appended so the user understands the dropdown is intentionally trimmed, not broken - The providers card surfaces ``models_total`` separately so the header still reads "397 models · OAuth" - A small "+N more" disclosure pill appears at the end of the rendered pill list (only fires for non-OAuth providers — OAuth cards never render pills) with a tooltip pointing at the slash command Featured selection rules ------------------------ Deterministic; same algorithm runs in both `/api/models` and `/api/models/live` so background enrichment doesn't undo the trim: 1. Always include the user's currently-selected model (sticky — no orphan IDs in the dropdown after a refresh) 2. Always include every entry from the curated static ``_PROVIDER_MODELS["nous"]`` list whose id maps onto a live id 3. Top up to 15 by walking ``_NOUS_VENDOR_PRIORITY`` round-robin (one model per vendor each pass) so no vendor monopolises the slots Changes by file =============== api/config.py - New `_format_nous_label` neighbour: `_NOUS_FEATURED_THRESHOLD = 25`, `_NOUS_FEATURED_TARGET = 15`, `_NOUS_VENDOR_PRIORITY` tuple, `_build_nous_featured_set()` helper (~80 LOC) - `_build_available_models_uncached` Nous branch: - Apply featured-set cap with sticky-selection signal - Return `extra_models` alongside `models` for the catalog tail - Decorate optgroup label with truncation count - Drop stale-4 fallback when authenticated but live-fetch empty (omit the group entirely; truth lives in the providers card and the next cache rebuild will heal it) - Keep stale-4 fallback when hermes_cli is unavailable (test envs, package mismatches) — that's a different failure mode - Detection symmetry: explicit `get_auth_status("nous").logged_in` check after the existing `list_available_providers()` loop, so the picker matches the providers card on hermes_cli versions where the two signals disagree api/providers.py:get_providers - Apply same featured-set cap so card body doesn't render 397 pills - Add `models_total` field reporting full catalog size (used by frontend for the "N models · OAuth" header text) api/routes.py:_handle_live_models - Apply same featured-set cap for `/api/models/live` so background enrichment via `_fetchLiveModels()` doesn't undo the dropdown trim - Use sticky-selection from `cfg["model"]["model"]` matching the main endpoint's logic static/ui.js:populateModelDropdown - Hydrate `_dynamicModelLabels` from `g.extra_models` so a selection outside the visible dropdown still renders with its proper label static/commands.js:_loadSlashModelSubArgs - Iterate `group.extra_models` so `/model` autocomplete covers the full catalog (not just the trimmed featured slice) static/panels.js:_buildProviderCard - Header count uses `p.models_total` (full catalog size) instead of `p.models.length` (trimmed slice) - Render trailing "+N more" disclosure pill when `models.length < models_total` with a tooltip pointing at the slash command static/style.css - New `.provider-card-model-tag-more` rule (italic, dashed border, cursor:help, no select) — visually distinct from real model pills Tests ===== `tests/test_issue1567_nous_picker_capacity_and_symmetry.py` (20 tests): - TestBuildNousFeaturedSet (8): unit tests on the helper — small-catalog no-op, large-catalog cap to target, disjoint+complete invariants, priority-vendor round-robin guarantee, sticky selection with and without `@nous:` prefix, curated-flagship preservation, empty-catalog handling, determinism - TestApiModelsLargeCatalog (2): /api/models cap behavior end-to-end on a synthetic 397-model catalog vs a 20-model catalog - TestNousDetectionSymmetry (2): picker includes Nous when `get_auth_status` agrees but `list_available_providers` disagrees; picker omits Nous when both disagree - TestNousLiveFetchEmpty (2): authenticated + empty-fetch omits group; hermes_cli unavailable still falls back to static-4 - TestProvidersCardPickerSymmetry (1): both endpoints agree on exactly the same featured-set IDs + total catalog count - TestFrontendExtrasContract (4): static-source assertions pinning the JS contract for `extra_models`, `models_total`, and the "+N more" disclosure Verified live on port 8789 (30-model catalog): - /api/models Nous group: provider="Nous Portal (15 of 30)", 15 models, 15 extra_models - /api/models/live?provider=nous: 15 entries (matches main path) - /api/providers Nous card: models_total=30, models=15 - Browser dropdown after backfill: 15 options, 30 entries in _dynamicModelLabels - Sticky selection: Claude Opus 4.7 (the active model) in the featured slice as expected 4073 pytest passed (was 4053 → 4073, +20 from this PR). 3 CI test runs (3.11/3.12/3.13) green. QA harness 11/11 passed. Reporter: Deor (Discord #report-bugs, May 03 2026 14:15 PT) Relayed by: AvidFuturist --- api/config.py | 237 +++++++- api/providers.py | 30 +- api/routes.py | 17 + static/commands.js | 9 + static/panels.js | 26 +- static/style.css | 10 + static/ui.js | 11 + ...e1567_nous_picker_capacity_and_symmetry.py | 555 ++++++++++++++++++ 8 files changed, 868 insertions(+), 27 deletions(-) create mode 100644 tests/test_issue1567_nous_picker_capacity_and_symmetry.py diff --git a/api/config.py b/api/config.py index 82c52398..f234addd 100644 --- a/api/config.py +++ b/api/config.py @@ -899,6 +899,122 @@ def _format_nous_label(mid: str) -> str: return f"{base} (via Nous)" +# Soft cap on how many Nous Portal models surface in the picker dropdown. +# Above this count, _build_nous_featured_set() trims the visible list to +# ~_NOUS_FEATURED_TARGET entries; the full catalog is still returned to the +# client under ``extra_models`` so /model autocomplete covers everything. +# Caps reflect human scannability — a 25-row dropdown is the practical UX +# ceiling, and per-vendor sampling at 15 keeps the flagship shape visible +# without one vendor dominating. +_NOUS_FEATURED_THRESHOLD = 25 +_NOUS_FEATURED_TARGET = 15 + +# Vendor-prefix priority order for featured selection. Lower index = picked +# earlier when sampling the live catalog. Reflects which vendors users have +# historically reached for first via Nous Portal (driven by the curated +# static list maintained in _PROVIDER_MODELS["nous"] and Discord feedback). +_NOUS_VENDOR_PRIORITY = ( + "anthropic", "openai", "google", "moonshotai", "z-ai", + "minimax", "qwen", "x-ai", "deepseek", "stepfun", + "xiaomi", "tencent", "nvidia", "arcee-ai", +) + + +def _build_nous_featured_set( + live_ids: list[str], + *, + selected_model_id: str | None = None, + target: int = _NOUS_FEATURED_TARGET, +) -> tuple[list[str], list[str]]: + """Trim a Nous Portal catalog into a (featured, extras) split. + + ``featured`` is what the picker dropdown renders. ``extras`` is everything + else — kept available so the slash-command `/model` autocomplete and the + ``_dynamicModelLabels`` map cover the full catalog. + + Selection rules (in order, deterministic): + + 1. Always include the user's currently-selected model if it's in the + catalog (preserves selection stickiness — no orphan IDs in the + dropdown after a refresh). + 2. Always include every entry from the curated static + ``_PROVIDER_MODELS["nous"]`` list whose id maps onto a live id — + those four are explicitly maintained as flagship picks. + 3. Top up to ``target`` by walking ``_NOUS_VENDOR_PRIORITY`` round-robin + (one model per vendor each pass) so no vendor monopolises the slot + budget. Within a vendor, the original ``live_ids`` order is preserved + — that's the order Nous Portal returned, which approximates recency. + + Returns ``(featured_ids, extras_ids)`` — both lists are subsets of + ``live_ids`` with disjoint membership and union equal to ``live_ids``. + + For catalogs ≤ ``_NOUS_FEATURED_THRESHOLD`` entries the function is a + no-op: ``featured == live_ids``, ``extras == []``. + """ + if not live_ids: + return [], [] + if len(live_ids) <= _NOUS_FEATURED_THRESHOLD: + return list(live_ids), [] + + chosen: list[str] = [] # preserves insertion order + chosen_set: set[str] = set() + + def _add(mid: str) -> None: + if mid and mid not in chosen_set: + chosen.append(mid) + chosen_set.add(mid) + + # Rule 1: sticky selection. Strip "@nous:" prefix if present so we can + # match against the live id space (which is bare "vendor/model"). + if selected_model_id: + sel = selected_model_id + if sel.startswith("@nous:"): + sel = sel[len("@nous:"):] + if sel in live_ids: + _add(sel) + + # Rule 2: curated flagships. Extract the bare ids from the static list + # entries (which are stored as "@nous:vendor/model"). + for static in _PROVIDER_MODELS.get("nous", []): + sid = static.get("id", "") + if sid.startswith("@nous:"): + sid = sid[len("@nous:"):] + if sid in live_ids: + _add(sid) + + # Rule 3: vendor-priority round-robin top-up. + by_vendor: dict[str, list[str]] = {} + for mid in live_ids: + if mid in chosen_set: + continue + vendor = mid.split("/", 1)[0] if "/" in mid else "" + by_vendor.setdefault(vendor, []).append(mid) + + # Walk vendors in priority order, then any leftover vendors alphabetically. + priority = list(_NOUS_VENDOR_PRIORITY) + leftover = sorted(v for v in by_vendor if v not in set(priority)) + vendor_order = priority + leftover + + # Round-robin: one model per vendor per pass until we hit the target or + # exhaust every bucket. + while len(chosen) < target: + added_this_pass = 0 + for vendor in vendor_order: + if len(chosen) >= target: + break + bucket = by_vendor.get(vendor) + if not bucket: + continue + _add(bucket.pop(0)) + added_this_pass += 1 + if added_this_pass == 0: + break # all buckets empty + + # Anything not chosen becomes extras (full-catalog completion surface). + extras = [m for m in live_ids if m not in chosen_set] + return chosen, extras + + def _apply_provider_prefix( raw_models: list[dict], provider_id: str, @@ -1767,6 +1883,22 @@ def get_available_models() -> dict: logger.debug("Failed to get key source for provider %s", _p.get("id", "unknown")) detected_providers.add(_p["id"]) _hermes_auth_used = True + + # Belt-and-braces: list_available_providers() is the primary signal + # for OAuth providers, but its `authenticated` field can disagree + # with `get_auth_status().logged_in` on some hermes_cli versions + # (the two fields are computed via different code paths). When the + # disagreement happens for Nous Portal, the Settings → Providers + # card renders the live catalog (because api/providers.py iterates + # all OAuth providers regardless of authentication state) but the + # picker dropdown comes up empty — a confusing asymmetry reported + # in #1567. Add Nous explicitly when get_auth_status agrees so the + # picker stays in sync with the providers card. + try: + if _gas("nous").get("logged_in"): + detected_providers.add("nous") + except Exception: + logger.debug("Failed to check Nous Portal auth status") except Exception: logger.debug("Failed to detect auth providers from hermes") @@ -2241,43 +2373,102 @@ def get_available_models() -> dict: } ) elif pid == "nous": - # Nous Portal exposes a curated catalog (~30 models, currently) - # via inference-api.nousresearch.com. Like ollama-cloud, we + # Nous Portal exposes a curated catalog (~30 models on most + # accounts, up to several hundred for enterprise tiers) via + # inference-api.nousresearch.com. Like ollama-cloud, we # live-fetch through hermes_cli.models.provider_model_ids() # rather than relying on the static four-entry list, which - # chronically drifts out of date (#1538). Fall back to the - # static list when hermes_cli is unavailable (test envs, - # package mismatches) so the picker is never empty. + # chronically drifts out of date (#1538). + # + # When the catalog exceeds _NOUS_FEATURED_THRESHOLD (~25) + # the picker dropdown gets a curated subset to stay + # scannable — the full list is still returned under + # "extra_models" for the slash-command autocomplete and + # the dynamic-label map (#1567). The optgroup label is + # decorated with the truncation count so users know more + # exists. raw_models = [] + extra_models: list[dict] = [] + truncated_label_suffix = "" + live_fetch_failed = False try: from hermes_cli.models import provider_model_ids as _provider_model_ids live_ids = _provider_model_ids("nous") or [] - raw_models = [ - # Prefix every live id with "@nous:" so routing matches - # the explicit-provider-hint branch of resolve_model_provider - # (same convention as the curated static list — see - # tests/test_nous_portal_routing.py for the invariant). - {"id": f"@nous:{mid}", "label": _format_nous_label(mid)} - for mid in live_ids - ] except Exception: logger.warning("Failed to load Nous Portal models from hermes_cli") + live_ids = [] + live_fetch_failed = True - if not raw_models: - # Static fallback: deepcopy so dedup/prefix mutation - # below does not bleed into the module-level catalog. + if live_ids: + # Sticky-selection signal: prefer the explicitly-active + # model from cfg["model"]["model"] (what the user is + # currently using) over cfg["model"]["default"] (the + # configured default suggestion). Falls back to the + # latter so first-load before any selection still works. + _model_cfg = cfg.get("model", {}) + _selected = ( + (isinstance(_model_cfg, dict) and _model_cfg.get("model")) + or default_model + or None + ) + featured_ids, extras_ids = _build_nous_featured_set( + live_ids, + selected_model_id=_selected, + ) + # Prefix every live id with "@nous:" so routing matches + # the explicit-provider-hint branch of resolve_model_provider + # (same convention as the curated static list — see + # tests/test_nous_portal_routing.py for the invariant). + raw_models = [ + {"id": f"@nous:{mid}", "label": _format_nous_label(mid)} + for mid in featured_ids + ] + extra_models = [ + {"id": f"@nous:{mid}", "label": _format_nous_label(mid)} + for mid in extras_ids + ] + if extras_ids: + # Show "(15 of 397)" so the user understands the picker + # is showing a featured subset, not a broken short list. + truncated_label_suffix = ( + f" ({len(featured_ids)} of {len(live_ids)})" + ) + elif not live_fetch_failed: + # Live-fetch returned an empty list AND did not raise — + # the user is gated as authenticated by detection above + # but the catalog endpoint replied with no models. + # Showing the static 4-entry curated list here would + # contradict the providers card (which always shows + # the live catalog) — exactly the asymmetry #1567 + # reports. Omit the Nous group entirely; the providers + # card already tells the truth, and a transient empty + # response will self-heal on the next cache rebuild. + logger.warning( + "Nous Portal authenticated but live-fetch returned empty — " + "omitting from picker (will retry on next cache rebuild)" + ) + else: + # hermes_cli unavailable / raised — fall back to the + # curated 4-entry static list so the picker is never + # empty in this degraded state. This matches pre-#1538 + # behaviour for environments without hermes_cli (test + # envs, package mismatches, isolated WebUI builds). raw_models = copy.deepcopy(_PROVIDER_MODELS.get("nous", [])) if raw_models: models = _apply_provider_prefix(raw_models, pid, active_provider) - groups.append( - { - "provider": provider_name, - "provider_id": pid, - "models": models, - } - ) + # Apply the same prefix transform to extras so /model + # autocomplete sees consistent IDs across the two lists. + extras = _apply_provider_prefix(extra_models, pid, active_provider) if extra_models else [] + group_entry = { + "provider": provider_name + truncated_label_suffix, + "provider_id": pid, + "models": models, + } + if extras: + group_entry["extra_models"] = extras + groups.append(group_entry) elif pid in _PROVIDER_MODELS or pid in cfg.get("providers", {}): raw_models = copy.deepcopy(_PROVIDER_MODELS.get(pid, [])) detected_models = auto_detected_models_by_provider.get(pid, []) diff --git a/api/providers.py b/api/providers.py index 74b41354..86825774 100644 --- a/api/providers.py +++ b/api/providers.py @@ -392,10 +392,19 @@ def get_providers() -> dict[str, Any]: pass models = list(_PROVIDER_MODELS.get(pid, [])) + models_total = len(models) # Nous Portal: prefer the live catalog so the providers card matches # the dropdown picker (#1538). Same fallback shape as the static-only # case below — when hermes_cli is unavailable or its lookup raises, # we keep the four-entry curated list. + # + # On large-tier accounts (#1567 reporter Deor saw 396 entries), we + # render the same featured subset the picker uses so the providers + # card body doesn't become a 396-pill wall. The full count is still + # reported via models_total — surfaced in the header line as + # "396 models · OAuth" by static/panels.js — so the user knows the + # complete catalog is reachable (via /model autocomplete or a future + # "show all" disclosure if added). if pid == "nous": try: from hermes_cli.models import provider_model_ids as _provider_model_ids @@ -403,12 +412,14 @@ def get_providers() -> dict[str, Any]: live_ids = _provider_model_ids("nous") or [] if live_ids: # Lazy-import to avoid circular dep with api.config. - from api.config import _format_nous_label + from api.config import _format_nous_label, _build_nous_featured_set + featured_ids, _extras = _build_nous_featured_set(live_ids) models = [ {"id": f"@nous:{mid}", "label": _format_nous_label(mid)} - for mid in live_ids + for mid in featured_ids ] + models_total = len(live_ids) except Exception: logger.debug("Failed to load Nous Portal models from hermes_cli") # Also include models from config.yaml providers section @@ -420,6 +431,13 @@ def get_providers() -> dict[str, Any]: models = models + [{"id": k, "label": k} for k in cfg_models.keys()] elif isinstance(cfg_models, list): models = models + [{"id": k, "label": k} for k in cfg_models] + # Recompute models_total when config.yaml contributes additional + # entries on top of the live/static catalog. For non-Nous + # providers models_total still equals len(models); for Nous + # we keep the live count (which already includes any models + # surfaced in the curated featured slice). + if pid != "nous": + models_total = len(models) providers.append({ "id": pid, @@ -430,6 +448,14 @@ def get_providers() -> dict[str, Any]: "key_source": key_source, "auth_error": auth_error, "models": models, + # models_total reflects the complete catalog size (e.g. 396 for + # an enterprise Nous Portal account), even when "models" is + # trimmed to a featured subset for UI scannability. The frontend + # uses this for the header text "396 models · OAuth" so users + # know the full catalog exists and is reachable via the slash + # command. For providers that don't trim, models_total == + # len(models) and the frontend behaves identically to before. + "models_total": models_total, }) # Scan custom_providers from config.yaml (e.g. glmcode, timicc) diff --git a/api/routes.py b/api/routes.py index 1c927620..e2c84492 100644 --- a/api/routes.py +++ b/api/routes.py @@ -4416,6 +4416,23 @@ def _handle_live_models(handler, parsed): if not ids: return _finish({"provider": provider, "models": [], "count": 0}) + # For Nous Portal, apply the same featured-set cap that + # /api/models uses so background enrichment via _fetchLiveModels() + # doesn't undo the dropdown trim — otherwise a 397-model catalog + # would still flood the picker after the initial render finished + # the cap. The full list is returned via the main /api/models + # endpoint's extra_models field for /model autocomplete; the live + # endpoint is purely a dropdown-enrichment surface, so it should + # match the dropdown's visibility budget. (#1567) + if provider == "nous": + try: + from api.config import _build_nous_featured_set + _default_model = (cfg.get("model", {}) or {}).get("model") if isinstance(cfg.get("model"), dict) else None + _featured, _ = _build_nous_featured_set(ids, selected_model_id=_default_model) + ids = _featured + except Exception: + logger.debug("Failed to apply Nous featured-set cap for /api/models/live") + # Normalise to {id, label} — provider_model_ids() returns plain string IDs. # For ollama-cloud use the shared Ollama formatter (handles `:variant` suffix). # For all other providers use a simpler hyphen-split capitaliser. diff --git a/static/commands.js b/static/commands.js index 375a9d67..6dce4c48 100644 --- a/static/commands.js +++ b/static/commands.js @@ -136,6 +136,15 @@ async function _loadSlashModelSubArgs(force=false){ const id=_normalizeSlashSubArg(model&&model.id); if(id) values.push(id); } + // Include extra_models (the catalog tail that doesn't render as + //