Imitation Learning via Reduction to ID Active Learning: A Comprehensive Survey

In this article, we propose a novel strategy called Dissimilarity Sampling (DS) to enhance the performance of adversarial reinforcement learning (ARL) agents in complex environments. The core idea is to encourage the agent to explore diverse perspectives by sampling from them based on their similarity to each other. By doing so, the agent can avoid acquiring redundant information and instead focus on discovering novel insights that lead to better decision-making.
To achieve this, we first calculate a score for each perspective based on the summed correlation between it and all other perspectives. These scores are then normalized to produce probabilities, which determine the next perspective to be sampled. By including stochasticity in the selection process, we prevent the agent from early-onsetting to an uninformative perspective and account for uncertainty in the estimated feature expectations.
We evaluate the DS strategy through experiments on a variety of environments, demonstrating its effectiveness in improving the agent’s performance. Our approach is robust against various adversarial attacks and can handle complex scenarios with multiple perspectives. By using discriminators that distinguish between expert and learner data for all available perspectives, we ensure that our DS strategy can adapt to changing environments and learn from diverse experiences.
In summary, Dissimilarity Sampling offers a powerful tool for ARL agents to explore complex environments more effectively, leading to improved decision-making and robustness against adversarial attacks. By sampling from perspectives based on their similarity, we enable the agent to focus on discovering novel insights and avoid redundant information, ultimately resulting in better performance in challenging tasks.