The developers were frightened by the power of the artificial intelligence model being developed

AI researchers warn of threat to humanity from advanced development

An American high-tech company specializing in artificial intelligence, OpenAI, is working on an advanced model that is so powerful that it has seriously alarmed the developers themselves.

Reports say the new Q* model has raised safety concerns and workers have voiced their concerns to the board of directors before CEO Sam Altman was fired, The Guardian writes.

Before Sam Altman was fired, OpenAI was reportedly working on an advanced system that was so powerful that it raised security concerns among company employees.

The artificial intelligence model caused such concern among some OpenAI researchers that they wrote to the board of directors before Altman was fired, warning that it could threaten humanity, Reuters reports.

The model, called Q* – and pronounced “Q-Star” – is capable of solving basic math problems it has never encountered before, according to tech news site the Information, which added that the pace of development of the system alarmed some security researchers. The ability to solve mathematical problems would be considered a significant development in artificial intelligence.

The reports follow days of chaos at San Francisco-based OpenAI, whose board fired Altman last Friday but then reinstated him Tuesday night after nearly all of the company’s 750 employees threatened to quit unless he was brought back. Altman also secured the support of OpenAI’s largest investor, Microsoft.

Many experts are concerned that companies like OpenAI are moving too quickly to develop artificial general intelligence (AGI), a term for a system that can perform a wide range of tasks at or above human intelligence and that could theoretically elude control person.

Andrew Rogoiski, from the Institute for Human-Centered AI at the University of Surrey, said having a large language model for solving mathematical problems (LLM) would be a breakthrough. He noted: “The LLM’s inherent ability to do mathematics is an important step forward in enabling AI to offer a whole new range of analytical capabilities.”

Speaking on Thursday last week, the day before his surprise dismissal, Altman indicated that the company behind ChatGPT had made another breakthrough.

Speaking at the Asia-Pacific Economic Cooperation (APEC) summit, he said: “Four times in OpenAI’s history, most recently just in the last couple of weeks, I’ve been in the room when we’ve kind of pulled back the veil of ignorance and we’ve moved forward by paths of discovery, and to do so is the professional honor of a lifetime.”

As The Guardian recalls, OpenAI was founded as a non-profit enterprise with a board of directors that runs a for-profit subsidiary run by Altman. Microsoft is the largest investor in commercial businesses. As part of the agreement in principle to bring back Altman, OpenAI will have a new board of directors chaired by Bret Taylor, the former co-CEO of software company Salesforce.

The developer of ChatGPT states that it was created with the goal of developing “safe and useful general purpose artificial intelligence for the benefit of humanity” and that the for-profit company would be “legally obligated to carry out the mission of a non-profit organization.”

The nonprofit’s emphasis on safety led to speculation that Altman was fired for compromising the company’s core mission. But his short-term successor as interim chief executive, Emmett Shear, wrote this week that the board “did not remove Sam from his position because of any specific disagreement on safety issues.”