TikTok Says AI Race Shifts From Model Scale to Production Optimization

TikTok said the decisive edge in today’s AI race is not owning the biggest model, but running AI that is tightly optimized for real-world production—where latency, cost, safety, and shifting user behavior matter as much as accuracy.

Speaking Thursday UTC at “Metacon 2026,” a major AI conference co-hosted by TokenPost in Seoul, Kanchan Sarkar, an engineering manager at TikTok (owned by ByteDance), argued that ‘general-purpose’ models are only a starting point. In a session titled “How TikTok Moves the Market in the AI Era,” Sarkar said the industry over-emphasizes model scale while underestimating the operational work required to make AI reliable at internet scale.

“A research model is built to maximize accuracy,” Sarkar said, contrasting lab benchmarks with production constraints. In live services, he noted, systems must absorb diverse inputs, handle thousands of queries per second (QPS), withstand adversarial abuse, and perform on domain-specific tasks that do not resemble public datasets. “Even at TikTok, applying open or general models as-is often fails to deliver the performance we need,” he added, describing optimization—along with techniques such as quantization—as essential steps to meet business requirements.

TikTok’s approach, Sarkar said, centers on a ‘teacher–student’ architecture. A large ‘teacher’ model can best capture the complexity of real environments, but it is typically too slow and expensive to serve at scale. TikTok instead transfers the teacher’s capabilities to a smaller ‘student’ model through methods including ‘feature distillation,’ ‘soft learning,’ and task-specific fine-tuning. According to Sarkar, the resulting student models can approach teacher-level performance while costing roughly 10 to 15 times less to operate in production.

He framed this as a pragmatic rebuttal to the idea that choosing between open-source and closed models is sufficient. “Open source and closed models alone are difficult to deploy directly in real services,” Sarkar said, emphasizing that domain-specific work is unavoidable. Fine-tuning, reinforcement learning, and retrieval-augmented generation (RAG)—a method that grounds model outputs by pulling relevant information from external sources—were highlighted as tools to achieve service-grade quality and reduce failures such as hallucinations.

Sarkar also stressed that AI deployment begins with choosing the right modality rather than forcing a single model to solve everything. Depending on the problem, the service may require text, audio, or image models—or combinations of them—each shaped by infrastructure constraints such as latency budgets and compute cost. On the infrastructure side, he said, teams must weigh ‘model compression,’ distillation, and hardware acceleration together rather than treating them as independent optimizations.

Crucially, TikTok believes offline evaluation is not enough. Sarkar described online experimentation as the “only” reliable validation method in consumer-scale services. Even if a model performs well on public benchmarks, he said, outcomes can diverge sharply under real traffic and user behavior, making A/B testing a mandatory gate before wider rollout.

Looking ahead, Sarkar outlined six technical and governance challenges he believes the industry still must solve: ‘continual learning’ to keep up with changing environments; stronger long-context understanding for lengthy videos and documents; improved ‘logical reasoning;’ scalable agent-based AI systems; ‘compliance by design’ to embed regulatory requirements into systems from the start; and ‘multimodal reasoning’ that integrates spatial and temporal signals across modalities.

Metacon 2026 runs July 3–4 in Seoul at COEX’s Grand Ballroom under the theme “AI Makers Rise,” bringing together companies and builders to share implementation experiences across AI technology, enterprise transformation, marketing, and investment. TikTok’s message underscored a broader industry shift: as models commoditize, durable advantages may increasingly come from optimization, systems engineering, and disciplined production testing rather than from raw parameter counts.