An image from MetaAI’s FaceBook page describing the release of OPT-175B to researchers.

Meta Opens Its Voluminous Language Model OPT-175B to Outside Review

While many companies have been criticized for their closed-door development of language models like GPT-3 because of dangerous built-in flaws, Meta (formerly FaceBook) is releasing its version to researchers–even welcoming input, according to a story on MIT’s technologyreview.com.

Meta, the multinational technology conglomerate that owns numerous social media companies, as well as  AI, VR, financial services and information services companies, is sharing its Open Pretrained Transformer (OPT-175B) model with 175 billion parameters trained on publicly available data sets to academic researchers, and is inviting their input, according to information on the MetaAI site.

“We strongly believe that the ability for others to scrutinize your work is an important part of research. We really invite that collaboration,” says Joelle Pineau, a longtime advocate for transparency in the development of technology, who is now managing director at Meta AI.

Misinformation, racism, sexism and toxic language have been among the problems that have cropped up in early language models, which have been integrated into several applications. Meta is seeking to ameliorate those issues and get feedback from government and educational institutions on how to improve the model and methodology.

“It’s a great move,” says Thomas Wolf, chief scientist at Hugging Face, the AI startup behind BigScience, a project in which more than 1,000 volunteers around the world are collaborating on an open-source language model. “The more open models the better,” he told technologyreview.com.

Meta is also releasing its code and a logbook that documents the training process, which contains daily updates from members of the team about the training data: how it was added to the model and when, what worked and what didn’t. In more than 100 pages of notes, the researchers log every bug, crash, and reboot in a three-month training process that ran nonstop from October 2021 to January 2022.

read more at technologyreview.com