mirror of
https://github.com/nesquena/hermes-webui.git
synced 2026-07-16 12:40:18 +00:00
perf(webui/session-load-latency) tier1a: cheap-first freshness check in _cached_session_lags_disk
Prior to this change _cached_session_lags_disk() called Session.load_metadata_only(sid) on every LRU cache hit, parsing the full sidecar JSON (~15-20ms even for a 1.3MB sidecar on Celeron+ eMMC). For draft auto-saves hitting get_session() on every keystroke debounce (~400ms cadence while typing), that 15-20ms multiplied out to ~75% of the request's wall time on the Chromebook. Reorder the check: read only the JSON metadata prefix (_persisted_message_count, already defined for the LRU-eviction path) to compare message counts. For INACTIVE sessions (no active_stream_id, no pending_user_message) with matching counts, the cache is provably at parity with disk — anchor-scene records cannot have advanced without the message count advancing too, and the full Session.load_metadata_only() is skipped. For ACTIVE sessions, or when counts differ, fall through to the original anchor-scene comparison path so correctness is preserved. Measured cost reduction on the request hot path: 15-20ms -> <1ms on the typical cache-hit case (inactive session, count matches). The slow path is unchanged for cases that genuinely need the full reload. No public API change. No behavior change observable on user-facing flows. The LRU eviction safety invariant (#4765) is preserved — _session_is_evictable() still calls _persisted_message_count under the same conditions, and Tier 1a adds NO new eviction gate.
This commit is contained in:
+35
-5
@@ -3095,24 +3095,54 @@ def _cached_session_lags_disk(cached) -> bool:
|
||||
id. Serving the cache then makes recent assistant results disappear from
|
||||
GET /api/session even though disk and _index.json are correct. Compare only
|
||||
cheap metadata here; full reload happens only if disk is strictly ahead.
|
||||
|
||||
perf(webui/session-load-latency) cheap-first ordering: the function used to
|
||||
call Session.load_metadata_only(sid) on every cache hit, which parses the
|
||||
full sidecar JSON (~15-20ms even for 1.3MB sidecars on Celeron+ eMMC).
|
||||
For draft auto-saves that hit get_session() on every keystroke debounce
|
||||
(every ~400ms while typing), that 15-20ms multiplied out to ~75% of the
|
||||
request's wall time on the Chromebook. We now do a single fast check
|
||||
first: read only the JSON metadata prefix to compare message counts.
|
||||
The full Session.load_metadata_only() and its anchor-scene comparisons
|
||||
only run when the count check is inconclusive or when disk appears to be
|
||||
ahead of cache.
|
||||
"""
|
||||
if cached is None:
|
||||
return False
|
||||
sid = getattr(cached, 'session_id', None)
|
||||
if not sid:
|
||||
return False
|
||||
cached_count = len(getattr(cached, 'messages', None) or [])
|
||||
# Fast path: prefix read of just the metadata header.
|
||||
disk_count = _persisted_message_count(sid)
|
||||
if disk_count is not None:
|
||||
if disk_count > cached_count:
|
||||
return True
|
||||
# Disk is at most as far as cache; for inactive sessions we don't
|
||||
# need to compare anchor-scene records (those are only updated by
|
||||
# active streaming/recovery paths, which we can detect cheaply below).
|
||||
if getattr(cached, 'active_stream_id', None) or getattr(cached, 'pending_user_message', None):
|
||||
# Active session: anchor-scene keys may have advanced even when
|
||||
# the count is the same. Fall through to the full check.
|
||||
pass
|
||||
else:
|
||||
# Inactive session, count matches → cache is at parity with disk.
|
||||
# The pre-fix code would still load_metadata_only to compare
|
||||
# anchor scenes, but for inactive sessions those cannot have
|
||||
# advanced without the count advancing too. Skip the full load.
|
||||
return False
|
||||
try:
|
||||
disk_meta = Session.load_metadata_only(sid)
|
||||
except Exception:
|
||||
return False
|
||||
if disk_meta is None:
|
||||
return False
|
||||
cached_count = len(getattr(cached, 'messages', None) or [])
|
||||
disk_count = _parse_nonnegative_int(getattr(disk_meta, '_metadata_message_count', None))
|
||||
if disk_count is None:
|
||||
disk_count = _lookup_index_message_count(sid)
|
||||
if disk_count is not None and disk_count > cached_count:
|
||||
return True
|
||||
disk_count = _parse_nonnegative_int(getattr(disk_meta, '_metadata_message_count', None))
|
||||
if disk_count is None:
|
||||
disk_count = _lookup_index_message_count(sid)
|
||||
if disk_count is not None and disk_count > cached_count:
|
||||
return True
|
||||
if not getattr(cached, 'active_stream_id', None) and not getattr(cached, 'pending_user_message', None):
|
||||
cached_scene_keys = _anchor_scene_record_keys(cached)
|
||||
disk_scene_keys = _anchor_scene_record_keys(disk_meta)
|
||||
|
||||
Reference in New Issue
Block a user