American social news aggregation and discussion forum Reddit has practically banned major search engines and AI (Artificial intelligence) from its platform. But it has offered exceptions to those who pay.
Which means, that if consumer utilize sites like Microsoft Bing, Duck Duck Go, Mojeek Quant or any other search engine, which is independent of Google’s indexing (browse reddit by utilizing site:reddit.com) then the consumer won’t be able to see results from the previous week.
As per 404 media, Google is the sole search engine, which displays results, when consumers browse on reddit.
Notably, Google has given $60 million to reddit to utilize its data for training its AI model.
As per the report, old results are still shown for alternative search results. But they are not able to crawl reddit for fresh content.
Confirming this new development, Microsoft representative have announced ‘Bing stopped crawling Reddit after they implemented their updated robots.txt file on July 1, which prohibits all crawling of their site.’
What Reddit Says?
This development comes around the time, when Reddit has just started to take action against those companies, who scrap data for their AI models. In a previous month, the company has just initiated update for its site’s robots.txt file to implement the policy against web scrapping (refers to a process of utilizing bots to acquire content & data from website).
The aim of this policy is to stop search engine from indexing and showing its posts and comments unless they have paid partnership.
Moreover, reddit has also declared, that this decision is not connected with their new partnership with google.
Speaking about this point with the verge, Reddit Spokesperson Tim Rathschmidt said, ‘This is not at all related to our recent partnership with Google.’
In addition, the spokesperson also declared that they have had discussions with several search engines. They haven’t able to establish any agreement with many of them, as some of them are not able to or haven’t agreed to give any promises regarding site’s use of content, which also include the use of AI.