TikTok Developing US-Specific Version Of Core Algorithm, Sources Reveal

TikTok is developing a separate version of its recommendation algorithm for its 170 million U.S. users. This effort aims to create a system independent of its Chinese parent company, ByteDance, in response to concerns from American lawmakers who are advocating for a ban on the app, according to sources familiar with the matter. This initiative, […]

by Drishya Madhur - May 31, 2024, 11:00 am

TikTok is developing a separate version of its recommendation algorithm for its 170 million U.S. users. This effort aims to create a system independent of its Chinese parent company, ByteDance, in response to concerns from American lawmakers who are advocating for a ban on the app, according to sources familiar with the matter.

This initiative, ordered by ByteDance in late 2023, preceded legislation requiring the sale of TikTok’s U.S. operations, which gained traction in Congress and was signed into law in April. The sources, who spoke on condition of anonymity due to the confidentiality of the information, indicated that splitting the code could potentially set the stage for divesting the U.S. assets, although no such plans are currently in place. The company has previously stated that selling its U.S. assets would be impossible.

TikTok initially declined to comment, but later responded to the story on X, stating, “The Reuters story published today is misleading and factually inaccurate,” without specifying what was incorrect. TikTok also quoted its federal lawsuit, saying, “the ‘qualified divestiture’ demanded by the Act to allow TikTok to continue operating in the United States is simply not possible: not commercially, not technologically, not legally. And certainly not on the 270-day timeline required by the Act.” Reuters defended its reporting.

In May, TikTok and ByteDance filed a lawsuit in U.S. federal court to block the law that mandates the sale or ban of the app by January 19. A U.S. appeals court has expedited the consideration of legal challenges to this new law.

In recent months, hundreds of engineers from ByteDance and TikTok in both the U.S. and China have been tasked with separating millions of lines of code. Their goal is to create an independent code base for TikTok’s recommendation algorithm, distinct from the one used by ByteDance’s Chinese version, Douyin, while removing any connections to Chinese user data.

This technical separation, which has not been previously reported, reveals the extent of TikTok’s efforts to mitigate the political risks it faces. U.S. President Biden and other proponents of the law argue that TikTok allows Beijing excessive access to data that could be used for espionage or influence over U.S. users.

Previously, Reuters reported that selling the app along with its algorithms is unlikely, as the Chinese government added content recommendation algorithms to its export-control list in 2020, making any divestiture subject to administrative licensing procedures. ByteDance initially developed TikTok’s recommendation engine in China and customized it for various global markets, including the U.S.

The challenge of separating the code, described by sources as tedious “dirty work,” highlights the difficulty of disentangling TikTok’s U.S. operations from its Chinese parent. This work is expected to take more than a year to complete.

Despite challenging the U.S. law in court on First Amendment grounds, engineers continue to work on separating TikTok’s U.S. recommendation engine from ByteDance’s broader network. An earlier effort to silo off U.S. user data, known as Project Texas, failed to satisfy U.S. regulators and lawmakers. Now, the company is intensifying its efforts to demonstrate the independence of its U.S. operations from its Chinese owner.

At one point, TikTok executives considered open sourcing parts of the algorithm or making it available for others to access and modify to show technological transparency. Executives have discussed the code-splitting project during team meetings and through internal communication platforms, as noted by sources who attended these meetings or viewed the messages. Reuters could not independently verify these internal messages.

Legal and compliance issues complicate the task of determining which parts of the code can be included in the new code base. The goal is to create a separate source code repository for a recommendation algorithm exclusively for TikTok U.S., ensuring it operates independently from other regions and from Douyin. However, this move might cut TikTok U.S. off from the engineering support of ByteDance in Beijing, potentially impacting the performance of the recommendation engine.