China has made a significant leap in artificial intelligence with the launch of DeepSeek, a series of Large Language Models (LLMs) developed by High-Flyer, a Chinese hedge fund. The latest version, DeepSeek-v3, introduced in December 2023, has surpassed its predecessors and outperformed other Chinese AI models while being significantly more cost-effective.
DeepSeek’s rapid development began in April 2023, when High-Flyer established its AI lab dedicated to advancing large-scale AI models. The DeepSeek-v3 model, built using a Mixture-of-Experts (MoE) approach, has demonstrated remarkable efficiency. It has been pre-trained on 14.8 trillion tokens and consists of 671 billion parameters, with 37 billion activated per token. The training process relied on Nvidia’s H800 GPUs, consuming 2.78 million GPU hours, which is far lower than Meta’s Llama 3.1, which required 30.8 million more GPU hours.
One of DeepSeek’s most significant breakthroughs is its ability to match OpenAI’s O1 model in multiple metrics while maintaining a drastically lower operational cost. The model has achieved a 93% reduction in cost per API call, making AI services more affordable for businesses and developers. Additionally, DeepSeek’s architecture allows it to run efficiently on high-end local computers, reducing reliance on cloud services. Optimized memory efficiency and batch processing further enhance its cost-effectiveness.
Despite its technological advancements, DeepSeek-R1 operates under China’s strict censorship policies. While this restricts certain discussions, it has not stopped its growing popularity. In fact, DeepSeek-R1 has become the most downloaded AI app in the U.S. and ranks third in India’s productivity category.
China’s push for AI dominance is not new. In March 2023, Baidu launched Ernie Bot, which was seen as China’s response to OpenAI’s ChatGPT. Within a day of its release, Ernie Bot recorded 30 million user sign-ups. However, it faced heavy criticism for dodging politically sensitive topics, such as President Xi Jinping, the Tiananmen Square crackdown, and human rights concerns related to Uyghur Muslims. These limitations raised doubts about China’s ability to create globally competitive AI models.
One of the techniques helping AI models like DeepSeek improve efficiency is knowledge distillation. This process involves transferring knowledge from a large, complex AI model (the “teacher”) to a smaller, faster model (the “student”). It helps reduce model size, making AI more accessible for devices with limited processing power. Distillation enhances inference speed, reduces latency, and minimizes memory usage. Several techniques, including logit matching, feature map transfer, and hint training, are used to refine the student model’s learning process. However, knowledge distillation has limitations, as the student model may not always match the full capabilities of the larger model.
With the growing demand for more efficient AI models, the competition in artificial intelligence is set to intensify. DeepSeek’s advancements indicate China’s strong push toward AI leadership, and as technology continues to evolve, global players will need to keep pace.
