Superintelligence

A superintelligence is a hypothetical agent that possesses intelligence surpassing that of the brightest and most gifted human minds. "Superintelligence" may also refer to a property of problem-solving systems (e.g., superintelligent language translators or engineering assistants) whether or not these high-level intellectual competencies are embodied in agents that act in the world. A superintelligence may or may not be created by an intelligence explosion and associated with a technological singularity.

University of Oxford philosopher Nick Bostrom defines superintelligence as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest".^[1] The program Fritz falls short of this conception of superintelligence—even though it is much better than humans at chess—because Fritz cannot outperform humans in other tasks.^[2]

Technological researchers disagree about how likely present-day human intelligence is to be surpassed. Some argue that advances in artificial intelligence (AI) will probably result in general reasoning systems that lack human cognitive limitations. Others believe that humans will evolve or directly modify their biology to achieve radically greater intelligence.^[3]^[4] Several future study scenarios combine elements from both of these possibilities, suggesting that humans are likely to interface with computers, or upload their minds to computers, in a way that enables substantial intelligence amplification.

Some researchers believe that superintelligence will likely follow shortly after the development of artificial general intelligence. The first generally intelligent machines are likely to immediately hold an enormous advantage in at least some forms of mental capability, including the capacity of perfect recall, a vastly superior knowledge base, and the ability to multitask in ways not possible to biological entities. This may allow them to — either as a single being or as a new species — become much more powerful than humans, and displace them.^[1]

Several scientists and forecasters have been arguing for prioritizing early research into the possible benefits and risks of human and machine cognitive enhancement, because of the potential social impact of such technologies.^[5]

^ ^a ^b Bostrom 2014, Chapter 2.
^ Bostrom 2014, p. 22.
^ Pearce, David (2012), Eden, Amnon H.; Moor, James H.; Søraker, Johnny H.; Steinhart, Eric (eds.), "The Biointelligence Explosion: How Recursively Self-Improving Organic Robots will Modify their Own Source Code and Bootstrap Our Way to Full-Spectrum Superintelligence", Singularity Hypotheses, The Frontiers Collection, Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 199–238, doi:10.1007/978-3-642-32560-1_11, ISBN 978-3-642-32559-5, retrieved 2022-01-16
^ Gouveia, Steven S., ed. (2020). "ch. 4, "Humans and Intelligent Machines: Co-evolution, Fusion or Replacement?", David Pearce". The Age of Artificial Intelligence: An Exploration. Vernon Press. ISBN 978-1-62273-872-4.
^ Legg 2008, pp. 135–137.

[FOOTNOTEBostrom2014Chapter_2-1] Bostrom 2014, Chapter 2.

[FOOTNOTEBostrom201422-2] Bostrom 2014, p. 22.

[3] Pearce, David (2012), Eden, Amnon H.; Moor, James H.; Søraker, Johnny H.; Steinhart, Eric (eds.), "The Biointelligence Explosion: How Recursively Self-Improving Organic Robots will Modify their Own Source Code and Bootstrap Our Way to Full-Spectrum Superintelligence", Singularity Hypotheses, The Frontiers Collection, Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 199–238, doi:10.1007/978-3-642-32560-1_11, ISBN 978-3-642-32559-5, retrieved 2022-01-16

[4] Gouveia, Steven S., ed. (2020). "ch. 4, "Humans and Intelligent Machines: Co-evolution, Fusion or Replacement?", David Pearce". The Age of Artificial Intelligence: An Exploration. Vernon Press. ISBN 978-1-62273-872-4.

[FOOTNOTELegg2008135–137-5] Legg 2008, pp. 135–137.

[1]

[2]

[3]

[4]

[5]