Transferring knowledge as heuristics in reinforcement learning: A case-based approach

Reinaldo A.C. Bianchi, Luiz A. Celiberto, Paulo E. Santos, Jackson P. Matsuura, Ramon Lopez De Mantaras

Research output: Contribution to journalArticlepeer-review

54 Citations (Scopus)


Abstract: The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain. A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms.

Original languageEnglish
Pages (from-to)102-121
Number of pages20
JournalArtificial Intelligence
Publication statusPublished - Sept 2015
Externally publishedYes


  • Case-based reasoning
  • Reinforcement learning
  • Transfer learning


Dive into the research topics of 'Transferring knowledge as heuristics in reinforcement learning: A case-based approach'. Together they form a unique fingerprint.

Cite this