A determined group of new age ideologues who believe artificial intelligence will become conscious and wipe out humanity are having an increasing amount of influence over U.S. AI policy.
This week, a State Department funded report declared that AI posed an existential threat akin to nuclear weapons—and even recommended potential jail time for unauthorized AI development. Last week, VentureBeat reported about a mutiny brewing at the National Institute of Standards and Technology (NIST), after staff discovered a leading AI doomer who predicts a “50/50” chance of AI wiping out humanity was being fast tracked into a role at the AI Safety Institute (ASI), a newly-formed body to help research and develop policy and guidelines for regulating the technology.
The ASI was created by NIST as part of Biden’s executive order on AI. The order itself was heavily shaped by the RAND Corporation, which Politico reports has become increasingly entwined with a movement promoting AI doom narratives. This includes taking $25 million in funding from Open Philanthropy, a charity of Facebook billionaire Dustin Moskovitz and key funder of AI doomerism.
ADVERTISEMENT
These events point to a broader pattern of influence over the U.S. government by people espousing ideas of effective altruism (EA), a movement that purports to believe in using evidence and reasoning in order to do the most good for the most people. Concerns about AI gaining sentience and ending humanity has become its focus in recent years, much to the dismay of early supporters like Steven Pinker.
While many are reluctant to use that label due to controversy around its extensive funding links to Sam Bankman-Fried and his fraudulent crypto exchange FTX, there are many ways to tell when someone subscribes to EA ideology.
The recent State Department funded report bears a striking resemblance to the heavy-handed proposals of the EA movement. Despite report co-author Edouard Harris's insistence he is not one, his years long presence on EA forum LessWrong and his previous collaboration with Eliezer Yudkowsky’s Machine Intelligence Research Institute (MIRI), suggests he at least shares many of the movement's views. The nature of his work with MIRI is unclear. Harris didn’t respond to a request for comment to clarify.
Yudkowsky is loud voice amongst AI doomers. Last year, critics claimed that he proposed nuclear strikes against countries defying a theoretical AI development treaty in a Time op-ed—though he later denied the allegations in a post on X.
Then there is the RAND Corporation, which helped consult and guide President Biden’s AI executive order. The consulting firm saw effective altruists take key roles in its executive leadership in recent years including its CEO Jason Matheny. This has led to accusations within the company of it pushing an EA agenda.
There is also Paul Christiano, the man being fast tracked into the AI Safety Institute, who is a key figure in EA and AI safety research circles. Christiano is less coy about his views: seeing odds of AI eradicating humanity as a coin flip. It was these views that contributed to the controversy of his purported appointment to the AI Safety Institute.
These connections hint at a wide web of EA affiliated figures influencing the direction of U.S. AI policy, and these ideas seem to be catching on throughout the Biden Administration.
In December, Chuck Schumer hosted a senate forum on AI. He opened with a speech about prospects about AI doom, in attendance was MIRI CEO Malo Bourgon who provided a written statement along with other invited attendees. Schumer would pepper attendees with questions about their “P(Doom)” number, an EA term for estimates on AI wiping out humanity.
A month prior, FTC head Lina Khan stated her P(Doom) as 15 percent. Secretary of Commerce Gina M. Raimondo—who was responsible for fast-tracking Paul Christiano—publiclychastized proponents of open AI development as irresponsible.
Schumer promised $10 million dollars to the ASI, while its umbrella org NIST struggles with funding issues. Meanwhile, the links between EA, RAND and the ASI have drawn scrutiny from some Democrats and Republicans, who have demanded more transparency.
The release of the State Department report, which cost taxpayers $250,000, appears to have doubled as a promotional effort for a new political PAC called American’s for AI Safety. Its goal: to pass stringent AI regulations by the end of 2024, aiming to make it a key issue in the coming election. The anti-AI crusade continues.
Editor's Note: This story has been updated to correct that the RAND Corporation drafted President Biden's executive order on AI, and MIRI CEO Malo Bourgon provided a written statement and responded to questions from lawmakers.
