Dario Amodei, CEO, and his sister Daniela, President, are the founders of Anthropic, which they hope will help make AI more understandable. (Source: Future of Life YouTube video)

Anthropic’s Goal: Finding Ways to Implement Safeguards into AI Systems

Former OpenAI VP of research Dario Amodei established Anthropic less than a year ago to research how to make AI more reliable and understandable. A $580 million investment into the company will help it reach that goal, according to a techcrunch.com story.

You see Anthropic is built to try and understand exactly how AI does what it does. And that is enough to give one pause. How does AI work? Really? Yes, it seems that even the folks that build popular algorithms are not sure how they do what they do exactly.

“With this fundraise, we’re going to explore the predictable scaling properties of machine learning systems, while closely examining the unpredictable ways in which capabilities and safety issues can emerge at-scale,” said Amodei in the announcement.

Daniela Amodei, President of Anthropic, previously worked as VP of Safety and Policy at OpenAI.

His sister Daniela, with whom he co-founded the public benefit corporation, said that having built out the company:

“We’re focusing on ensuring Anthropic has the culture and governance to continue to responsibly explore and develop safe AI systems as we scale.”

Both of the Amodeis used the word “scale” and that raised a flag with the article’s author, Devin Coldeway:

“There’s that word again — scale. Because that’s the problem category Anthropic was formed to examine: how to better understand the AI models increasingly in use in every industry as they grow beyond our ability to explain their logic and outcomes.”

The company has already published several papers looking into, for example, reverse-engineering the behavior of language models to understand why and how they produce the results they do. GPT-3, for instance, is the best-known language model, but its internal operations are a mystery even to its creators.

Models operate without clear reasoning, which is a problem when they make errors, such as being biased in recognizing faces (especially erring in recognizing black ones) and tend to be sexist when describing professions that are dominated by men.

It is a popular opinion that humans will never be able to completely understand how AI thinks and processes. With that in mind, one can’t help but wonder: has Skynet already won and we are just at the beginning of a Schwarzenegger movie? Hopefully Anthropic can help answer that question.

read more at techcrunch.com