Roskomnadzor is trying to block the OpenAI search program

Roskomnadzor sent a letter to a number of hosting providers about the identification of the GPTBot search robot from OpenAI. The agency recommends blocking the bot’s ability to view and analyze Internet pages to “prevent the collection of information about critical resource vulnerabilities” that are within the companies’ area of ​​responsibility. Experts note that if such algorithms are banned, businesses will have to resort to other methods of collecting relevant data, for example, by disguising their bots.

“Kommersant” has read the letter from the Main Radio Frequency Center (GRFC, subordinate to Roskomnadzor) with recommendations for identifying and blocking the OpenAI bot GPTBot, which the department sent out to a number of hosting providers on December 11. The letter states the need to assess the risks of collecting information about the vulnerability of resources or “other sensitive information, including those containing personal data.” If such risks are identified, the GRCH indicates the need to block the bot’s requests. The department also sent instructions on how to do this.

An employee of the duty shift of the center for monitoring and managing the public communications network of the GRChTs confirmed by telephone to Kommersant that the letter had been sent. Roskomnadzor declined to comment. Hosting providers Beget and Rusonyx did not answer “Kommersant”. The Coordination Center for .RU/.РФ domains reported that they had not received the letter.

Previously, Qrator Labs experts reported (see Kommersant on December 6) that in November of this year the company noted a sharp increase in the activity of malicious bots. First of all, online stores were attacked during active sales. During the period November 24–30, 40.7% of all anonymous bot calls to the industry last month occurred. The largest single attack was on November 26, when the company recorded 5.2 million requests. According to Qrator Labs, the number of requests to Russian resources from GPTBot set a record high.

OpenAI launched this bot in August to crawl the web and collect open data, which it will later use to improve its ChatGPT neural network. Shortly after the launch, in the wake of criticism of the product and the company for unauthorized data collection, OpenAI published instructions on how to block a bot’s access to a site or provide it with only partial data.

“It’s no secret that AI bots can be used not only for peaceful purposes, but also serve the interests of cybercriminals. Based on such solutions, they are already creating tools for quick hacking, phishing, and ensuring the operation of darknet sites,” Nikita Tsaplin, general director of the hosting provider RUVDS, told Kommersant.

Mr. Tsaplin notes that attention needs to be paid to these risks, and therefore the recommendation of Roskomnadzor makes it possible to limit the possibilities of automated search for vulnerabilities using certain AI bots: “They are becoming more and more effective and, therefore, more dangerous. Of course, distinguishing a bot from a regular user is not always easy, but in general this work needs to be done. We regard the initiative as sound, and we can recommend it to all webmasters.”

“GPTBot itself is designed as correctly as possible, it always “introduces itself” when it appears on the network, and declares compliance with the access rules for bots set by resource owners,” explains Qrator.AntiBot product manager at Qrator Labs Georgy Tarasov. The expert notes that OpenAI independently posted instructions on how to deny the bot access or limit its work to certain pages. Additionally, the company provides a list of IP addresses from which requests originate for verification purposes.

“If companies on the RuNet and on the global Internet continue to deny access to GPTBot and other AI search robots at the same pace as is happening now, then AI / ML businesses will have to resort to other methods of collecting relevant data,” Mr. Tarasov believes. For example, disguising bots as legitimate users and purchasing aggregated data from bot farm owners, and this already falls into the category of unwanted bot attacks.”

Tatiana Isakova, Alexey Zhabin