Modes and authorization
Observe and Recommend help build an understanding of workload behavior. Admission and automatic scaling require both global authorization and a suitable module mode. Writes are disabled by default.
Scheduled overrides
Set fixed resources or replicas for defined business windows, including start, end, timezone, priority, and covered fields. Other fields follow their normal policies; normal calculation resumes after the window.
Schedules use explicit IANA timezones and storage stays in UTC. The platform defaults to UTC+8 and pages display UTC+8. Never write converted display times back as storage values.
Degradation and recovery
Each field follows bypass → schedule → eligible model → module fallback → unchanged. Values must remain valid, fresh, and within bounds. OOM floors apply outside bypass.
Record reasons for insufficient history, stale inputs, model failure, or missing eligible forecasts. Consecutive healthy cycles are required before recovery to avoid unstable switching.
When manual intervention is needed
- Enable Bypass and record a reason.
- Disable the corresponding global setting and return to Recommend or Observe.
- Review snapshots, action records, and actual resources.
- Fix inputs or policy and compare results before gradually restoring authorization.
Bypass and global settings do not restore earlier resource or replica configurations automatically. Consider current service state and action history when restoring them.
Configure OOM memory protection separately
Modules separate OOM protection from prediction metrics. Configure lookback, memory uplift, and heap pressure, then connect OOM counts, pre-OOM working set, memory limits, and JVM heap metrics. Rules missing their required inputs do not apply, and recommendation details identify the missing inputs.
OOM score weighting is disabled by default. When enabled, it penalizes memory scores only where memory was underallocated in the evaluation window; CPU scores are unaffected. Memory floors provide protection while scores compare models.
Hold, degrade, and recover during an outage
The Operator tracks each module separately. A connection fault does not immediately switch replicas to fallback. Configure timing in system settings; the platform distributes it. The following describes defaults.
| State | Replica behavior |
|---|---|
| Normal | Follow platform decisions within module bounds, cooldowns, and steps. |
| Cached | Use a decision less than ten minutes old for scale-up only, without retaining urgency. |
| Holding | Without valid cache, retain current replicas for five minutes without immediately applying fallback. |
| Degraded | After the hold, move toward fallback in steps, only upward, with degradation cooldown between actions. |
| Recovering | Follow upward decisions; scale-down requires both two consecutive successful responses and at least five minutes of stable recovery. |
Configure decision and module cache lifetimes, hold, fallback cooldown, recovery responses, and stability through operator.decision_cache_ttl_seconds, module_cache_ttl_seconds, degrade_hold_seconds, degrade_cooldown_seconds, recover_answers, and recover_stable_seconds, all under the operator. prefix. Expired module configuration also holds replicas. An explicit no-decision answer outranks cached advice.
The decision cache is in memory and does not survive restarts. Offline action reports can be persisted separately; see offline reporting.