Thinking as an Action

Abstract

We propose a novel architecture to build an Artificial General Intelligence (AGI) in a virtual environment. To experiment with curiosity we use as a reward in a reinforcement learning (RL) algorithm the cosine similarity between recent thoughts and past thoughts as sentences given by a large language model (LLM). The agent can decide, using the Bellman equation to act as a standard agent, by moving, jumping, performing a task, observing and thinking. Observing and thinking is the process of modifying its inner dialogue by given a representation of the environment to a LLM and reflecting on its past thoughts which will consequently change its predicted Q values and decision making. We have developed an experimental intelligent agent which interacts with the open source Minetest video game as a virtual environment.