“Network of City Portals” prohibited “Yandex” from using its content for neural networks

Skhulev Media Holding banned part of its media assets from search robots, including those from Yandex (MOEX: YNDX), use your content for the operation of neural network products. The change occurred on the eve of the launch of the Yandex Neuro service, which allows users to get answers to questions without visiting sites. Experts note that site owners are not yet clear about the benefits of the service or the tactics for working with it, but they are well aware of the risk of losing traffic.

Administrators of the “Network of City Portals” sites (E1.ru, NGS.ru, “Fontanka”, etc., owned by Skhulev Media Holding) wrote a prohibitive directive in the robots.txt files for the YandexAdditional search robot, Kommersant discovered. Thus, the sites prohibit Yandex from using their content as a source of answers for neural network products – in particular, for the Neuro service presented on April 16. The restrictions, according to Kommersant’s observation, were set between February and now; similar directives apply to robots from OpenAI (GPTBot and ChatGPT-User). Another site of Skhulev Media Holding, Psychologies.ru, also imposed a ban on YandexAdditional; he already prescribed a similar restriction for GPTBot in February.

Corporate Marketing Director of Skhulev Media Holding Timur Tokurov clarified to Kommersant that the bans were written only on some of the sites and “as part of an experiment.” “During surveys that Yandex conducted before the launch of Neuro, many publishers expressed a desire to be able to control this use. We are still in dialogue,” he added. Ambassador of Internet sites in Yandex search Mikhail Slivinsky told Kommersant that those publishers who decided to exclude answers from Neuro “do this for A/B testing” (comparing the effectiveness of two variants of changes by deploying them simultaneously) .

The output in Neuro is formatted as follows: first, a carousel of buttons-links to sources of information is displayed, and why – the result of neural network generation. The developer of the ProZen analytics extension, Igor Torkhov, says that in the Copilot service (built into Microsoft Bing, in Russia not available without a VPN), links to sources, in his opinion, are more conducive to transitions: “Firstly, they are located at the end, – secondly, they are accompanied by the text “learn more.”

Publishers who impose bans on the use of content in neural network products “believe that the established scheme of “traffic in exchange for announcing content” is breaking, and are afraid of its further decline,” explains Vladimir Kharev, head of search optimization at Finam. According to him, market participants had already experienced a drop in traffic when launching “quick replies,” several years before those in Yandex began to be compiled using neural networks. At the same time, in his opinion, “Neuro” is unlikely to significantly affect the volume of distributed traffic: “On the one hand, it looks like a development of “quick answers”, and on the other, like an experiment in answering queries that were previously answered by searches. basically answered poorly.”

The benefits of launching Neuro for Yandex and its users are obvious, “but why webmasters need it is still an open question,” emphasizes Timur Ugulava, co-owner of Mediasfera Group of Companies. In the future, site owners may agree to “exchange” part of the search traffic for increased media coverage, he argues: “Right now, I don’t yet see brands being actively mentioned in Neuro results.” SmartSEO director Sergei Koshkin adds that online store owners can make decisions using a different logic: “It may be beneficial for stores that information from the catalog forms the basis of the text and is then accompanied by a link.”

Yuri Litvinenko