Ours residual:可保真复现与可消融的代码架构¶
目标与边界¶
ours 不是一个新的 manipulation 算法。它在 Studio 的 PHC-X scene、case、asset、contact
label、reset、common rollout、evaluation 和 renderer 接口上,复现已审计 residual 的
policy、observation、action 与 reward 语义。
第一目标是一份 ours baseline config;第二目标才是受控 ablation。一个 ablation 只能改变 明确命名的一项配置字段,并保留父配置、输入/输出维度、配置哈希和全部 feature/reward diagnostics。
Studio 始终拥有运行时接口;参考 task、loader、asset/contact sidecar、renderer 和 evaluator 绝不进入 target runtime。
三层职责¶
flowchart TB
I["canonical case/result"] --> M["input.py\nmaterialize once per case"]
S["Studio PHC-X task\ncase/assets/contact/reset/PhysX"] --> O["ours formulas\npolicy / observation / reward"]
O --> H["one PHC host class\naction to Studio PD + Studio reward buffer"]
H --> S
R["Studio rl-games\nCommonAgent / player / checkpoint"] --> O
C["baseline config / ablation config"] --> O
M --> S
R --> S
O --> V["independent formula checks + live trace"]
1. Studio runtime layer¶
submodules/PHC/phc/env/tasks/humanoid_im_studio_residual.py remains the only class that
touches PHC task internals. It is responsible only for:
- obtaining Studio current/reference body state, humanoid DOF state, object link state/qpos/qvel, active joint names/mask, contact labels/regions/forces, timeline and local asset geometry;
- passing the existing Studio tensors directly to the three formula functions;
input.pychecks static names, shapes, units and frame counts while loading the canonical case; - applying the semantic core's normalized humanoid action through the existing Studio PHC-X
pre_physics_step, and writing the returned scalar reward into Studio's reward buffer; - accumulating only update-level reward scalars and buffering the small opt-in verification trace.
It must not contain feature concatenation, reward formulas, residual masks, hidden configuration constants, or a second implementation of audited residual math.
2. pipeline/physics/ours/ semantic core¶
Keep this deliberately small. The old catch-all task.py is removed rather than renamed or
exploded into a directory tree. Modules are named after the method's fundamental
concepts, not after implementation plumbing:
pipeline/physics/ours/
input.py # canonical inputs -> Studio native inputs and case-local config record
policy.py # teacher action and residual action composition
observation.py # 2596D formula and named slices
reward.py # reward formula and named terms
rlgames.py # the three framework-required model/agent/player subclasses
train.py # inherited-environment PHC command and one phase runner
run.py # public `physics.method=ours` entrypoint
policy.py, observation.py and reward.py own the ours method semantics. They are the
only files allowed to define their respective formulas. There is no parallel runtime-state schema:
the PHC host already owns the live state and calls the formulas with natural tensor groups.
input.py owns one-shot canonical materialization, the case-local resolved-config record, and
returns the materialized directory after strict validation; it is not imported on the PHC hot path. Small private helpers are preferable
to thin dataclasses, custom exception hierarchies or one-file-per-term abstractions.
The semantic core has no import from either reference or vendored reference package. Formula parity is established by test vectors only; all live runtime classes are Studio classes.
3. Configuration and runner layer¶
configs/methods/physics/ours.yamlis the baseline configuration. Controlled ablations are separate, named YAMLs that inherit the baseline and change only their declared fields; there is noprofile_idand no loose semantic Hydra knob.- The public YAML is the only semantic configuration authority. The run root contains one complete resolved copy and its hash; no second schema, revision protocol or compatibility translation is maintained.
train.pypasses the resolved values explicitly to PHC Hydra because the vendored learning YAML is only structural host glue. It startssys.executableand inherits the caller's environment; device visibility belongs to the shell or scheduler.submodules/PHC/phc/run_hydra.pybecomes a one-line registration call intopipeline.physics.ours.rlgames.rlgames.pymust not importinput.py,train.py,run.pyor a backend; it contains only the three subclasses required to attach Studio RMS, reward scalars and output finalization to the existing rl-games interfaces.
Baseline config before ablations¶
The baseline is source-informed but runs wholly in Studio. Source checkpoint provenance and source normalizer state are outside the Studio runtime contract; source residual checkpoints are not loaded.
The baseline config must lock all of the following at once:
| Contract | Baseline-config rule |
|---|---|
| Observation | named 2596D layout, nonzero Studio-derived bbox when enabled, source feature gates and segment order |
| Action | all source-selected channels, source clip/scale, exact normalized-action/PD ordering |
| Reward | every source term and original additive/multiplicative graph, evaluated from Studio tensors |
| PPO/player | Studio CommonAgent/player/checkpoint interfaces, with all semantically matching current-reference PPO numbers recorded in ours.yaml; no reference custom agent/player path |
| Timeline | explicit teacher/object reference convention with a same-frame trace proof |
| Articulation | Reuse Studio active-joint name/link/DOF/qpos mapping. The baseline 2596D layout requires exactly one active joint; loading a multi-active case fails rather than silently reducing it. |
| Fixed-object physics | Studio fixed-base asset, gravity disabled, joint damping/friction 20/5, and the named Studio 2-mm PhysX contact envelope; every field is persisted in the resolved contract/hash |
Studio-owned contact labels and geometry feed the source reward formula through one documented
conversion. Studio canonical data supplies boolean hand2 labels [T,2]; reward.py directly
materialises a float Studio reference tensor [T,52] by copying the left/right bit only to the
five corresponding Thumb3/Index3/Middle3/Ring3/Pinky3 bodies. The other 42 canonical bodies,
including both wrists and proximal finger links, remain exactly zero. The enabled
hand/other-body and part-specific terms consume that named 52-body tensor, its exact body-name
map, live [N,52] body-to-active-region distances and live [N,30]
finger-to-active-region distances. The adapter must fail if either the canonical hand2 labels,
the documented Studio mapping, or geometry is absent; it must not load a sidecar or zero-fill a
missing input.
This preserves formula, threshold, coordinate and action/PD semantics while retaining Studio
runtime ownership. A read-only audit of the selected completed b004/b005/b009 jobs proved that
the reference converter uses exactly this ten-distal-tip scatter, that its motion [331:383] and
VLM sidecar contain the same values, and that the current Studio hand2 files equal the converter's
recorded source hand2 files. Those source files remain audit evidence only and are never runtime
inputs. Because every selected reference non-hand label is zero, its cg_other reference branch is
neutral; the target reproduces that behavior without fabricating geometric labels.
Named slices and reward terms¶
Every observation block and reward term has a stable identifier, enabled gate, source contract, shape, units and diagnostics. Examples:
obs.body_778
obs.object_links_30
obs.dof_current_153
obs.dof_velocity_153
obs.dof_reference_153
obs.teacher_action_153
obs.active_joint_1
obs.raw_fingertip_force_30
obs.finger_object_pose_450
obs.finger_wrist_pose_450
obs.finger_contact_40
obs.wrist_object_pose_60
obs.object_bbox_144
reward.rb, reward.ro, reward.rcg,
reward.part, reward.part_vel, reward.handle_coarse, reward.handle_normal,
reward.progress, reward.feet_slide, reward.dof_limit, reward.action_rate
For the fullbody layout the mandatory sequence is 778 + 30 + 153 + 153 + 1 + 153 + 153 + 1 +
30 + 450 + 450 + 40 + 60 + 144 = 2596. observation.py rejects a config unless enabled
observation blocks sum to exactly 2596.
The PHC-X teacher keeps Studio's native local-root SMPL self observation, while the residual
policy's 778D block uses the reference-equivalent localRootObs=False encoding. The live adapter
computes both views separately from the same Studio body tensors; it never reuses the teacher view
as the residual block or changes teacher state/action ownership.
reward.py computes the final reward and every permitted named term. rig is an internal neutral
factor at the zero-weight baseline and is neither computed nor logged. Studio's existing TensorBoard path
records the rollout/update mean of the final reward and every named term; it does not write raw
per-step tensors or introduce a second diagnostic pipeline. This makes an ablation a declarative
config-YAML change rather than an edit to a monolithic task function.
The phase-progress reward keeps physical initialization and reward supervision distinct. Studio
resets the object to the canonical case q0 with zero object DOF velocity. The source-equivalent
reward origin is nevertheless q_ref[max(current_phase_start, episode_start)], and the signed
opening/closing fraction is clamped to [0,1]. A live reset qpos must not replace that canonical
reference origin, even when reconstruction noise makes q_ref[0] differ from case q0.
Verification gates¶
No baseline or ablation config may enter PPO training before all relevant gates pass:
- reference configuration and enabled algorithm contract are recorded;
- independent formula checks compute expected observation blocks, composed action, each reward term and final graph without calling the implementation under test a second time;
- live Studio trace uses non-synthetic asset/link/contact/timeline tensors and records physics, reference lookup, teacher-input and episode-start frame indices independently;
- Studio CommonAgent/player rollout and one PPO update use the selected baseline PPO contract;
- a short Studio smoke from fresh Studio initialization completes before any long partial or full training.
Every case-specific checkpoint, smoke, video, closed-loop, train-phase and evaluation record
carries the exact input/case/config identity. Formula-only and PHC-X-teacher audits explicitly
declare scope=case-independent and are bound to the same resolved config hash; they do not
pretend to consume a case. These audit records are verification outputs, not runtime handoffs.
run_ours() directly loads one canonical case, materialises the required Studio backend files,
validates them, and starts the partial run. All case-specific checks may reuse the same
studio_input/ directory.
Reference capture is run directly from a separately activated read-only audit environment; target
verifiers only read its bounded trace and never rewrite Python/library/device environment variables.
Closed-loop parity replays only the captured 153D residual-action rows for at most 64 steps through
a normal case-local Studio checkpoint/player. It never copies reference actor weights or RMS into a
Studio checkpoint, and the replay path is unavailable unless the bounded trace output is enabled.
pipeline.physics.ours.__init__ must expose public helpers lazily (or remain empty). A dotted
import such as pipeline.physics.ours.policy must not eagerly import run.py, a backend, or
input.py inside the PHC child process.
All parity captures, config hashes and opt-in per-frame traces live under the run output root, not in source trees. The training hot path performs no per-step JSON or NPZ writes. Long-run gate and resume policy lives in standalone operational scripts, not in the semantic runtime. An ablation report must name its parent config and the one changed contract.
Confirmed design decisions¶
- Runtime method, configuration and output identity is
ours. Controlled ablations are named config YAMLs, not runtime profiles. - Source residual checkpoints and source normalizer state are never loaded by Studio.
- Baseline human residual action is 153D with
residual_start=0,residual_scale=0.5andresidual_clip=1.0. The object receives no residual action; Studio's existing PD execution remains authoritative. - The baseline raw 30D fingertip-force observation slot is explicitly zeroed. The separate 40D finger-contact feature remains enabled and is computed from Studio contact/force tensors.
- The baseline 144D object-bbox feature is enabled and constructed from Studio asset metadata. A case without a valid bbox is rejected before training; it is never silently zero-padded.
- The baseline reward is the complete reference-v2 multiplicative/additive graph, evaluated only
from Studio tensors. It replaces the current Studio parent-reward composition; every weight is
an
ours.yamlfield for controlled YAML ablations. - The selected comparison runs were launched on 2026-08-01 with tracked reference HEAD
4edd949a36ad7b4e6e008ed2cbadca52eaf6b1a5. Three surviving pre-launch dirty runtime files are strong mtime/hash-supported candidates for the launch overlay, and their hashes are frozen inoutput/verification/shared_migration_20260801/selected_reference_runtime_provenance.json. The launcher saved neither a source snapshot nor a Git diff, so these candidates must not be described as a complete or bit-exact reconstruction of the historical dirty overlay. Their reconstructed effects are the articulated-object self-collision filter, optional canonical frame-zero object q0/zero qvel, and 1200-epoch checkpoint cadence; they do not replace the committed reward formula. Later August-4 reference edits are audit history, not the formula oracle for those checkpoints. In particular, those later edits added finger bodies tor_pand changedhandle_coarsefrom the semantic handle region to the whole collision surface; neither edit belongs to this baseline. - Baseline
ig=0is strictly neutral:rigis exactly1, and the runtime does not compute, log, or use an interaction-graph tensor as a reset/termination predicate. A future positive-igsemantic ablation requires an independent namedoursYAML and the complete Studio geometry/contact inputs; it must not silently change this baseline lifecycle. - Region, geometry and contact metadata are always loaded from Studio. No reference sidecar
loader or fallback exists in
ours. Every Studio case must expose the metadata required by the enabled reward contract; this includes canonical boolean[T,2]hand labels plus the exact named ten-distal-tip[T,52]conversion, its PHC humanoid body-name mapping,[N,52]body-center-to-region distances and[N,30]finger-body-to-region distances for the part-specific terms.cg_otherconsumes the live full-body physics contact signal; its selected-reference non-hand labels are all zero, exactly as produced by the audited converter. Any missing required metadata, mapping, or geometry fails normal input loading and is never hidden by changing reward weights. oursreuses Studio's existing asset, active-joint, qpos-reference, contact-label and contact-region loading/mapping methods. It receives only their validated tensors and metadata; the method layer adds observation, policy and reward formulas but has no asset/data loader.- The baseline
oursconfig follows the fixed 2596D single-joint contract, like CoDA's one-target reward. Multi-active cases are rejected before training; native multi-joint support is deferred to a separate future extension and is not silently reduced. This means exactly one canonical active joint, not that the Studio object asset may contain only one joint: cases with passive object joints retain their complete Studio qpos/qvel state, reset, physics and recorder columns. The adapter resolves the declared canonical active joint by name to its Studio target DOF; it never selects a first column by position. - Teacher state and action come only from Studio's integrated PHC-X tracking interface.
oursdoes not import a reference PHC-X task, PNN runtime or teacher checkpoint; its residual policy uses the selected network/action-composition design and trains from Studio initialization. - Studio's canonical timeline is the only timebase for
oursobservation and reward references.oursdoes not add a reference-enginet+1offset; teacher action uses only the timing defined by Studio's tracker. Per-step traces record state, reference and teacher frame indices. - The baseline residual policy uses the reference-v2
1024 -> 1024 -> 512construction, action distribution initialization and residual-composition semantics: separate actor/critic ReLU MLPs, actor-mean constant-zero initialization, fixedlog_sigma=-2.9, and no learned sigma. Thus initial deterministic evaluation preserves the Studio PHC-X base action, while training samples use the fixed exploration standard deviationexp(-2.9). All tunable values areoursconfig fields; architecture or scale ablations require a separate named YAML. - The policy input is the fixed reference-v2 2596D segment order documented above. Its implementation is a thin assembly adapter over existing Studio state, reference, local-frame, contact and bbox interfaces; it must reuse those helpers rather than reimplementing asset, geometry, tracking or PHC-X logic. Its two policy object slots are explicitly the live articulated actor root followed by the live active-child rigid body. In particular, the first slot is not substituted with a URDF root rigid body that can carry a fixed local transform; this preserves the reference policy's base-slot coordinate meaning while retaining Studio as the runtime state owner.
- The PPO baseline copies every current-reference numerical setting whose meaning is identical in
Studio
CommonAgent; epsilon-greedy and DAgger-KL are not carried over because they are disabled in the selected reference configuration. In particular, the selected reference's effective CLI launch replicates the one Studio canonical case into512collection environments, usesepisode_length=300with native early termination, and collects512 * 32 = 16384samples per PPO epoch as one 16,384-sample minibatch. The case YAML'snumEnvs: 2048is overridden by the recorded reference launch command and the reference TensorBoardinfo/epochsstep increment confirms 16,384 frames per epoch. These are collection numerics, not mixed-case sampling. Fixed player evaluation remains one environment at Studio frame zero for the canonical 150-frame trace and disables early termination. Training uses the selected run's executed0.30 mpelvis termination threshold: its generatedterminationHeight: 0.15field is shadowed by a hard-coded0.30in the task implementation. In Studio this is measured above the canonical physical ground plane,live_root_z - canonical_ground_height < 0.30; this is identical to the source expression for its zero-ground world and remains correct for canonical cases with a nonzero vertical world origin. PPO ablations use separate namedoursYAMLs. - The baseline reward weights are exactly the selected fullbody-v2 values:
p=30,r=1.5,pv=.25,rv=.02,op=5,or=.1,opv=.1,orv=0,ig=0,cg_hand=5,cg_other=5,cg_all=3,cg_finger=5,eg1=2e-5,eg2=2e-5,eg3=1e-11,part=6,part_vel=.25,progress=2,handle_coarse=4,feet_slide=.25,dof_pos_limits=-1,action_rate=-.005,hand_pos_reward_weight=.25,hand_velocity_reward_weight=.20andhand_rot_reward_weight=.25. The selected-runr_paverages over exactly 21 key bodies and applies.25only to its two wrists; it does not append the 30 finger bodies and it divides by 21 rather than renormalizing by the weight sum. Its all-52-body linear-velocity term uses the legacy.20hand mask, while rotation and angular velocity use.25. Contact, part and handle rewards remain active. - The selected reward graph is exactly `rb * ro * rig * rcg * r_part * r_part_vel * r_handle_coarse * r_handle_normal
- r_progress + r_feet_slide + r_dof_pos_limits + r_action_rate`. Its baseline enables part-specific contact/progress: it replaces only the generic all/finger contact subterms and generic progress, while preserving hand, other-body and contact-energy subterms. All required active-part/handle region, geometry and contact values come from Studio interfaces.
r_handle_coarseuses Studio's named active contact-region point cloud, which is the coordinate-converted equivalent of the selected run's semantic handle points. It does not grant coarse credit for approaching an arbitrary point on the door/drawer surface. The separate reference-body proximity weights and object reset use the Studio packaged physics URDF collision surface (512 fixed/root samples plus 1024 active-link samples), transformed through Studio named live/reference link states. None of these terms uses renderer visual geometry. The live contact graph threshold isany(abs(force_component) > 0.1 N); the separate 40D fingertip observation retains its own clipped-force-norm>0.2 Nmask.- The selected part-contact distance is
0.02 m(handlePartDistin the audited baseline);reward.pyapplies it to both full-body and finger active-region gates. r_handle_normalis neutral in the baseline because the selected fullbody-v2 config gives it no positive weight. A future named YAML may enable it only when Studio supplies collision geometry, surface normals and the required contact mapping; otherwise loading fails rather than returning a neutral fallback.- Normal
oursinput loading uses Studio interfaces to require exactly one active joint; its resolvable DOF, child-link and qpos/qvel reference; aligned reference frames; active-part/contact-region and finger/body-label mappings; valid bbox metadata; and every required observation/reward shape and unit. Any failure aborts the run: it never reads a reference sidecar, zero-fills, or disables a reward to proceed. - Studio's PHC motion export is tracker-local in XY. During the one-time
input.pymaterialisation,oursreads the case-local Studio PHC object's namedtracker_local_xy_originand writes a separate PHC-local articulated reference by subtracting it from only object root/link world positions. This makes object physics, reference links and human-local observation features share the same Studio coordinate basis; orientations, velocities, qpos and all link-local collision/bbox geometry are unchanged. The generic articulated manifest remains the unmodified canonical-world record, and no external transform or source data is consulted. - Training reset uses Studio's native reference-state Hybrid initialization, configured to the selected
fullbody-v2 semantics (
hybrid_init_prob=0.1,hybrid_init_max_fraction=0.5): the probability selects a canonical frame-zero reference reset, while the complementary branch samples a Studio reference state from at most the first half of the motion while still leaving one complete configured rollout before the motion ends. Consequently a 150-frame canonical motion with a 300-step rollout has frame zero as its only legal start. It never selects PHC's generic default/T-pose branch. This changes no runtime ownership: Studio provides reset, canonical timeline and PHC-X state. Deterministic smoke and evaluation reset at Studio reference frame zero. Object physics is reset only by Studio'sHumanoidImPassiveObject: the case-defined fixed root pose, the Studio case JSON's explicit initial qpos (q0) for every object joint, and zero root/DOF velocity are installed on every reset. The canonical object trajectory remains noisy pseudo-supervision for observation, reward and timeline diagnostics;oursnever writes its qpos/qvel into the simulated reset state. Phase-aware progress keeps this physical reset separate from reward supervision. Its reward origin is the canonical timeline valueq_ref[max(current_phase_start, episode_start)]; a live physical q0 must not replace that formula. The humanoid keeps the complete Studio motion state, including root and 153D DOF velocity, for bothHybridand deterministicStart. This intentionally does not reproduce a reference-loader boundary artifact that hard-codes only its frame-zero DOF velocity to zero while deriving its root/body velocities differently. Studio's PHC motion state is the canonical physical reset contract. - Fixed articulated furniture is a cold-start Studio physics contract: the root remains fixed,
gravity is disabled, all object joint velocities reset to zero, joint damping/friction are
20/5, and PhysX uses the namedours_contact2mm_sim.yaml2-mm contact envelope. These are Studio asset/simulator settings, not imported runtime code. They prevent passive joints from moving before contact and preserve the collision envelope used by the 2-cm part-contact reward. The resolvedoursconfig and checkpoint-compatibility hash include all four fields. - The three selected Studio cases are independent single-case runs. Each run creates and trains its own policy, optimizer, fresh RMS, checkpoint, TensorBoard stream and evaluation artifacts; there is no mixed-case sampling, shared policy or cross-case resume.
ours.yamlis one case-independent method configuration. Studio's existing single-case entry receives exactly one canonicalcase.json/result.ptpair per invocation; it derives the case-specific inputs and writes an isolated run directory. There are no case-specific method YAML copies or a case manifest insideours.yaml.- Each independent case run evaluates every 100 PPO epochs, retains a full Studio checkpoint
every 200 epochs, and also retains Studio's current best checkpoint. The baseline maximum is
1200 PPO epochs; cadence changes require a named
oursYAML. The 200-epoch partial is one continuous CommonAgent collector process, and the authorized 200-to-1200 continuation is one further continuous process. Evaluation-boundary snapshots are emitted by that process and evaluated separately; an evaluation must never be implemented by rebuilding the 512-env collector every 100 epochs, because that repeats its seed, Hybrid curriculum and environment state instead of continuing the selected PPO lifecycle. - Each case starts with a fresh 200-epoch partial. It automatically continues to the
1200-epoch full run only when canonical input loading, smoke and fixed Studio evaluation/video pass; no
non-finite value or anomalous termination occurs; TensorBoard reward terms and total reward are
healthy and rising; reward scale and convergence rate are comparable with the selected reference
at equal environment frames; and checkpoints, RMS and evaluation artifacts exist. A failed case
stops before full training and enters A/B debugging without affecting other cases.
Because GPU PhysX continuous net-contact-force readback is not bitwise repeatable, the partial
gate additionally requires at least three same-GPU, same-checkpoint Start/frame-zero raw evals.
It records median/min/max for reward, MPJPE, active-joint RMSE, opening and contact recall, and
saves video only for repeat 1. The final task threshold is diagnostic at epoch 200 because the
selected reference itself learns b004 after that boundary; at the final epoch-1200 audit,
--require-final-performancerequires the median and every repeat to pass with no threshold crossing. The full runner invokes that strict audit on the identity-boundfull/train_1200/ours.pthbefore it can write terminalfull_run.json=PASS; a failed median, any failed repeat, or a threshold crossing leaves the completed training artifacts but records terminal verification FAIL.scripts/physics/run_ours_training.shis the single post-pretraining operational command: it runs the partial, the repeated fixed-eval audit, the fail-closed partial gate, and the full continuation only when that verifier exits PASS. Its final argument is the integrity-bound case-to-reference baseline manifest. The semantic runtime does not import or fork verification scripts. - The source-relative terminal audit above is an integration/replication diagnostic, not the
project-level task-success definition. The current state machine in
docs/experiments/physics_metrics.mdis a DEV implementation snapshot, not a paper-frozen contract; articulation outcome, contact/full-horizon wrapper and thresholds remain pending human selection. Current proximity plus marginal-force telemetry must not be called verified same-link pair contact. Historical source rollouts without the chosen formal evaluation may only be used as explicitly labelled opening proxies when comparing absolute displacement and normalized-progress candidates. - Full20's lifecycle field
task_statusis the terminal opening gate, not the paper-level complete state-machine Binary Success. It uses the same target-owned signed-opening helper as offline Studio/source analysis: the origin is canonicalq_ref[0], the sign follows the maximum active-phase reference excursion, and opposite-direction displacement cannot count as opening. Evaluation validates but never rewrites the training-stage partial-gate record or its hash. - The frozen teacher is inherited only from Studio's existing
physics.phcxconfiguration and integrated PHC-X tracker.ours.yamlhas no duplicate teacher path. The residual policy starts fresh and has no residual-checkpoint, source-config-path or source-normalizer input. - Observation normalization has one
ours.yamlboolean,normalize_input. The baseline enables Studio's fresh running statistics; normal checkpoint resume restores only that Studio checkpoint's own state. The method has no external-RMS mode and does not add a custom snapshot or update cadence. It inherits the current StudioCommonAgent/rl-games model lifecycle: rollout is evaluated with the current case-local RMS, then the Studio RMS is updated by each PPO training forward. With the baseline's one 16,384-frame minibatch and six mini-epochs, a fresh one-update checkpoint therefore has RMS count1 + 6 * 16,384 = 98,305. This matches the selected clean reference's six-update cadence while retaining Studio's own RMS implementation and checkpoint restore path.