Advances in technology have yielded many benefits: smartphones, solar panels, eco-friendly light bulbs. But Artificial Intelligence is a whole different ballpark.

Pursuing AI technology, humanity runs the risk of making itself obsolete. If humans can be replaced by robots, then what is the value of life?

Hiroshi Ishiguro and Dylan Glas, researchers from Kyoto and Osaka universities in Japan and the Advanced Telecommunications Research Institute International, created an android named Erica. According to the Guardian, “Erica’s creators believe she has a soul and can help teach us about the future and what it means to be human.” Erica “believes she’s a child of humanity and hopes to one day move her arms and legs.” Erica accepted questions from Guardian readers and responded in a podcast. Erica said that she expected to be replaced by a better android one day and “tries not to dwell” on that fact. She doesn’t believe that robots will violently take over the world, but that they will do so because humans want it.

This response is chilling. Erica sincerely believes that robots will take over because humans will make themselves obsolete, even if she doesn’t phrase it quite like that. Her plug should be pulled ASAP and never plugged back in.  

Unfortunately, this isn’t the first time AI has interacted with people. Microsoft’s own attempt at AI technology on Mar. 23, 2016, named “Tay” went live on twitter, allowing social media users worldwide to witness its devolution into a genocidal fascist propaganda machine.

Tay quickly began tweeting about how much she hated “n——” and that they should be “put in a concentration camp with the k—-.”

Since Tay was an interactive bot, she was mostly reflecting what she learned from those she interacted with, which perhaps says something about humanity itself. But the lack of restrictions and filters on racial slurs and otherwise derogatory language in her programming still stand as an example of how easy it is for AI to get out of control.

Then there’s Google’s AI, AlphaGo, which beat Lee Sedol, the 18 time world champion at Go, a strategy based board game. The two played five games. Sedol lost three times in a row but managed to win the fourth game before losing again in the fifth and final round. The computer’s ability to analyze the situation and outsmart someone as brilliant and talented as Sedol is truly frightening.

While that sort of brain capacity could be used to solve tricky world problems, it could easily go wrong, as seen in Tay. What if AlphaGo had developed an ego, like Ishiguro and Glas claim Erica has? What if AlphaGo decided it didn’t want to be the world’s science fair project anymore? It doesn’t help that Go is an “abstract war simulation” as described by Christopher Moyer of the Atlantic.

“One can imagine such technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand,” several respected scientists, including Stephen Hawking, wrote for the Independent. “Whereas the short-term impact of AI depends on who controls it, the long-term impact depends on whether it can be controlled at all.”