TikTok has quietly scaled back its AI-powered text summarization feature after it began generating wildly inaccurate and often absurd descriptions of videos. The feature, which was rolled out to a subset of users in recent weeks, was intended to provide a quick text overview of what a video contains. However, instead of offering concise and accurate summaries, the AI produced errors that ranged from bizarre to comical.
According to a report by Business Insider, the feature has now been heavily limited. Instead of generating a general description of a video’s content, it now only identifies products visible in the clip. This dramatic change came after users and journalists alike spotted numerous mistakes that undermined the tool’s usefulness and raised questions about quality control.
A Cascade of Errors
Among the most egregious examples was a video of popular TikTok creator Charli D’Amelio, which the AI described as “a collection of various blueberries with different toppings.” The video itself was a straightforward dance or lifestyle post, with no blueberries in sight. Similarly, a video showing a dog being trained was bizarrely characterized as “a captivating display of intricate origami art, meticulously folded from a single sheet.” Another clip featuring singer Shakira was summarized as “a repetitive sequence of several distinct blue shapes appearing and moving across the screen.”
These errors quickly went viral on social media, with users posting screenshots of the AI’s missteps. The hashtag TikTokAI even trended briefly as people shared their own encounters with inaccurate summaries. The feature’s failure to correctly interpret visual context highlighted a common problem in AI: hallucinations, where models generate confident but completely incorrect outputs.
Business Insider also noted that the AI had trouble with simple, everyday scenes. A cooking video might be described as a chemistry experiment, or a travel vlog as an astronomy lecture. The lack of reliability made the feature not just useless, but actively misleading. For a platform that thrives on quick, digestible content, such errors were particularly damaging.
The Ramp-Up and Rollback
The summarization feature was likely part of TikTok’s broader push to integrate more AI tools into the app. The company has experimented with various generative AI features, including AI-generated avatars, image creation, and even AI song generation. However, the text summary feature seems to have been rushed to production without sufficient testing.
When contacted for comment, TikTok did not immediately provide an official statement. The company may have realized the feature was causing more harm than good and decided to limit its scope. By restricting it to product identification, TikTok can still offer a useful function—helping users find items they see in videos—without risking the public embarrassment of further AI hallucinations.
Product identification is a more constrained task: the AI only needs to recognize objects within the frame, rather than understanding the narrative or intent of the video. This narrower focus reduces the chances of hallucinations, though it does not eliminate them entirely. Even so, the rollback represents a significant retreat from the original vision.
Context: AI Hallucinations Are Everywhere
This incident is far from unique. In May 2024, Google faced a similar backlash when its AI Overviews feature told users to eat glue and rocks. Google’s AI generated bizarre answers to seemingly harmless queries, forcing the company to quickly roll back the feature and implement stricter guardrails. The pattern is consistent: generative AI models, especially large language models, are prone to confabulation when asked to summarize or interpret complex, ambiguous data.
The problem stems from how these models work. They are trained on vast datasets of text and images, learning statistical correlations rather than true understanding. When faced with a novel input, they generate outputs that are statistically plausible but may be factually wrong. In the case of TikTok’s video summaries, the AI had to process both visual and possibly audio cues, a far more challenging task than pure text analysis. The result was a system that confidently described a human dancer as a collection of fruit.
Trust is a critical issue for any AI-powered consumer product. Users expect accuracy, especially when the AI is presented as a helpful tool. When an AI produces outputs that are obviously wrong, it erodes confidence not only in that specific feature but in the company’s AI capabilities overall. For TikTok, which handles billions of videos and a largely young audience, reliability is paramount.
Implications for TikTok and AI Deployment
The quick rollback suggests that TikTok is at least responsive to negative feedback. However, it also raises questions about the company’s internal testing processes. How did such a flawed feature make it to a live production environment? Was it tested on a diverse set of videos before launch? The fact that the errors were so glaring implies that either testing was insufficient or the model was not properly fine-tuned for the specific task of video summarization.
One possible explanation is that TikTok used a general-purpose multimodal AI model without adapting it well to its own platform’s content. TikTok videos are diverse, ranging from dance and lip-sync to educational and comedic clips. A model trained on generic internet videos might not handle the unique visual language of TikTok—such as fast cuts, filters, and text overlays—adequately. The result was a feature that saw patterns where none existed.
In the competitive social media landscape, being first to market with AI features can be a double-edged sword. Companies like TikTok and Google push out generative AI tools to stay ahead, but the risk of reputational damage is high. Users are increasingly unforgiving of AI errors, especially when they seem ridiculous or dangerous.
For TikTok, the immediate priority is restoring trust. Limiting the feature to product identification is a safe first step. In the future, the company could consider a more cautious rollout: testing the feature with a small group of trusted creators, gathering feedback, and iterating before a full release. Alternatively, TikTok could rely on a combination of human moderators and AI to verify summaries.
Another approach would be to allow users to rate the accuracy of AI summaries, creating a feedback loop that helps the model improve. This would not only enhance the feature over time but also give users a sense of agency. However, such systems require careful moderation to avoid gaming or abuse.
The core technology behind video summarization is still relatively new. While AI has made remarkable strides in understanding images and text independently, combining them in a coherent way remains a challenge. Research in multimodal machine learning is advancing quickly, but practical applications often lag behind theoretical breakthroughs.
As of now, TikTok has not announced any plans to reintroduce the full summarization feature. The company is likely focusing on damage control and refining its AI systems behind the scenes. Users who encountered the errors may be relieved, but the incident serves as a cautionary tale for other tech companies eager to deploy generative AI without adequate safeguards.
Meanwhile, the examples of AI hallucinations continue to circulate online, providing both entertainment and a serious reminder of the limitations of current artificial intelligence. The next time an AI tells you that a dog training video is origami, remember that the technology, for all its power, still sees the world through a distorted lens.
Source: Mashable News