Abstract
If the intelligence of artificial systems were to surpass that of humans significantly, this would constitute a significant risk for humanity.
Time has come to consider these issues, and this consideration must include progress in AI as much as insights from the theory of AI. The papers in this volume try to make cautious headway in setting the problem, evaluating predictions on the future of AI, proposing ways to ensure that AI systems will be beneficial to humans – and critically evaluating such proposals.