摘要
Human-in-the-loop (HITL) reinforcement learning can improve exploration and policy acquisition by incorporating external guidance during training. In practice, however, guidance is often noisy, intermittent, and constrained by human effort, especially when intent is inferred from brain-computer interface (BCI) signals. This motivates a central question for reliable HITL-RL: how guidance should be used and when it should be discontinued. This paper presents NeuroRL, a simulation-based HITL-RL framework for studying imperfect BCI-derived intent guidance under controllable reliability. Electroencephalographic (EEG) classifier accuracies estimated from 28 participants parameterize probabilistic guidance, injected as corrective actions in GridWorld and preference-based feedback in Meta-World. We also introduce a warm-up-gated HITL stopping method that calibrates the transition from guided exploration to autonomous learning and activates post-handoff deterioration detection only after this warm-up phase, using Sustained Decline and Peak-Relative Decay signals. Experiments show that guidance reliability and intervention management jointly shape performance: GridWorld remains relatively robust under moderate noise, whereas Meta-World requires higher reliability for stable manipulation learning. Fixed-stop, dense-horizon, cross-accuracy, and intermediate-accuracy analyses show that useful stopping regions shift with guidance reliability and that prolonged imperfect guidance can introduce cumulative bias. Transfer and ablation studies indicate that the prediction-verification protocol identifies useful local stopping regions beyond the primary setting, and that the warm-up gate prevents premature deterioration detection during early exploration. These results support NeuroRL as a controlled framework for analyzing imperfect HITL guidance and a practical intervention management policy for handoff to autonomous reinforcement learning.