Finally, we show that our unsupervised agents can perform comparably to agents explicitly trained to work cooperatively, in some cases even outperforming them. We evaluate our approach in three different multi-agent environments where another agent's success depends on altruistic behaviour. We formalize this concept and propose an altruistic agent that learns to increase the choices another agent has by preferring to maximize the number of states that the other agent can reach in its future. Some concrete examples include opening a door for others or safeguarding them to pursue their objectives without interference. We propose to act altruistically towards other agents by giving them more choice and allowing them to achieve their goals better. Thus, it is beneficial to develop agents that do not depend on external supervision and learn altruistic behaviour in a task-agnostic manner. In the case of human agents, their goals and preferences may be difficult to express fully they might be ambiguous or even contradictory. However, explicit knowledge of other agents' goals is often difficult to acquire. Such an approach assumes that other agents' goals are known so that the altruistic agent can cooperate in achieving those goals. Can artificial agents learn to assist others in achieving their goals without knowing what those goals are? Generic reinforcement learning agents could be trained to behave altruistically towards others by rewarding them for altruistic behaviour, i.e., rewarding them for benefiting other agents in a given situation.