A Note Before We Begin

Over the past six months, I lost the motivation to keep updating this blog after a breakup. Later, I got pulled into my research group's work on image recognition and algorithm analysis, autonomous driving path planning, and intelligent vehicle control algorithms, which put my writing on hold entirely. Thanks to the encouragement of the Computer Science department head and the buzz around recent hot topics, I was asked to give a tech talk on large language models—and that finally reminded me to update this blog.

In this post, I'll cover three topics:

• From cloud computing power to on-device computing power.
• Data privacy in the AI era.
• Computing power demands in the AI era. The content covers the latest QwQ, large language model training methods, EXO's network-based computing power sharing, and global computing power statistics.

From Cloud Computing Power to On-Device Computing Power

Let's kick off this topic with Apple's product, Apple Intelligence.

In June 2024, at the Worldwide Developers Conference, Apple announced:

The personal intelligence system Apple Intelligence brings powerful generative models to iPhone, iPad, and Mac

Voice assistants are the most typical application of large language models. After ChatGPT went viral, Huawei, Xiaomi, and countless other companies rushed to train and release their own LLM-powered voice assistants. Apple, meanwhile, announced it would integrate GPT. However, due to mainland China's policies, Apple's AI features still can't be used properly in the mainland to this day.

Then this February:

Apple will integrate Alibaba's large language model and other AI services into iPhones and other devices sold in China. Apple announced it would adopt Alibaba's AI model for its products sold in the Chinese mainland.

CNBC

I'm sure everyone was baffled at the time. DeepSeek was all the rage—why didn't Apple choose DeepSeek, an open-source model, instead of Alibaba's Qwen series? When Apple made this decision, its stock even took a nosedive. NVIDIA's stock crashed precisely because of DeepSeek, so why would Apple make this call?

Before answering that question, let's turn our attention to DeepSeek. It's incredibly powerful. Its strengths are being open-source and lightweight—with just 671B parameters, it can go toe-to-toe with the world's most advanced closed-source models. But how much VRAM does it take to deploy?

A whopping 700GB! To deploy a DeepSeek R1 617B version with normal output speed, we'd need roughly 700GB of VRAM—about 10 A100 GPUs—costing around 2 million yuan.

Granted, that's already a massive reduction compared to what GPT-3.5 and GPT-4.0 required, because back then GPT could only run in the cloud, relying on the massive computing clusters behind OpenAI—something ordinary companies could only dream of. But fortunately, OpenAI's products drove a revolution in this field, paving the way for DeepSeek. And thanks to DeepSeek's open-source nature, more new models have been able to stand on the shoulders of giants. That's when QwQ was born!

On March 6, 2025, Alibaba's Qwen team released the QwQ open-source model in a big way—with only 32B parameters!

The mighty QwQ, with just 32B parameters (small by LLM standards), can match DeepSeek R1's full 671B version—and even surpass it! QwQ answers the question of why Apple chose Alibaba: for individual users, a single 4090 GPU is enough to run it, compressing the cost from 2 million down to 20,000. But is that really the case? The media has been hyping it up lately, but the truth is...?

The Alibaba team describes QwQ like this:

This is a model with 32 billion parameters whose performance is comparable to DeepSeek-R1, which has 671 billion parameters (37 billion of which are activated)

"QwQ-32B: Embracing the Power of Reinforcement Learning"

That's right—QwQ was only compared in performance to DeepSeek with 370B activated... or was it?

Architecture

What I didn't mention at the tech seminar was that QwQ was only compared in performance to DeepSeek with 370B activated. But this is exactly the major part I want to talk about: the architecture of DeepSeek and QwQ, which builds on its approach. (Since the seminar invited two non-specialist teachers and audience members who asked us questions, I had to keep things simple and cut a lot of details.)

Thanks to the Dense architecture, consumer-grade devices can run QwQ. DeepSeek, by contrast, uses a MoE architecture—or rather, a Hybrid MoE (Mixture of Experts) architecture. Based on the results, it might seem like Dense is the better choice? Not necessarily—we need to dig into the architectural details.

What's the difference between these two architectures?

Mixture of Experts (MoE) models work by splitting the model into multiple expert sub-networks and dynamically selecting the right experts to compute based on the input data. Each "expert" has strong processing power in a specific domain, and MoE intelligently picks the appropriate experts for each task. This mechanism keeps computational overhead low while significantly boosting the model's expressive power and flexibility. Especially with large-scale datasets, MoE models avoid redundant computation by precisely selecting different experts for specific tasks, effectively reducing resource consumption.

That long paragraph is a pain to read. In simple terms, the MoE architecture specializes AI functions—certain types of tasks are handled by designated "expert" modules. This is also the reason DeepSeek only activates about 370B parameters during inference.

Compared to MoE models, Dense models are the traditional deep neural network architecture. The design philosophy of Dense models is simple—every neuron (or computing unit) participates in every computation. Regardless of task difficulty, every parameter in a Dense model is involved in each calculation. This makes Dense models stable on relatively simple tasks, but when facing complex problems, they can struggle.

Because Dense models don't intelligently select computing units like MoE does, every training run requires computing and updating all parameters, which brings huge computational and storage demands. As a result, Dense models have higher computational costs, and efficiency drops significantly when handling large-scale datasets or complex tasks.

In simple terms, the Dense architecture throws all parameters at the problem—brute force wins the day.

Earlier I said that from the results, Dense seems to win. But from an architectural standpoint, MoE clearly has the edge. So why does QwQ, using Dense, outperform DeepSeek with MoE?

Let's start with performance requirements. MoE gains powerful reasoning by distributing tasks across expert models, but—and this is a big but—it demands extremely high-end hardware, requiring very strong parallel computing capabilities. Dense, on the other hand, is the exact opposite: even running on CPU + memory, its efficiency doesn't plummet.

MoE is like a pampered super-genius student, while Dense is the easy-to-raise adorable kid.

Overall, small-parameter models benefit from Dense for better quality, while large-parameter models use MoE for better efficiency.

Beyond the small parameter count making up for Dense's shortcomings, the more important factor is QwQ's training method.

Technical Deep Dive

The title of QwQ's release page reads: Embracing the Power of Reinforcement Learning

The Qwen team explains: recent research shows that reinforcement learning can significantly improve a model's reasoning ability. For example, DeepSeek R1 achieved state-of-the-art performance by integrating cold-start data and multi-stage training, enabling deep thinking and complex reasoning.

Since QwQ's technical report is still scarce, but it was trained following DeepSeek's approach, we can combine some of DeepSeek's training methods to understand why QwQ is so powerful.

QwQ-32B's training process has three stages: pre-training, supervised fine-tuning, and reinforcement learning. The reinforcement learning stage is further divided into two key phases, which is what allowed QwQ to surpass DeepSeek.

To understand why, let's break down: what is "cold start"? What is reinforcement learning? And what are the two key phases?

"Cold Start"

To help everyone understand cold start, let me use an everyday example.

During cold start, a large model is like a "child who knows nothing," constantly making mistakes, generating a bunch of illogical answers, and even falling into meaningless loops.

Using "cold-start data", in the early stage of AI training, we first fine-tune the model with a small batch of high-quality reasoning data—like giving the AI a "beginner's guide."

Referencing DeepSeek's optimized cold-start steps:

• Generate data from large models – Researchers use few-shot prompting.
• Generate data from DeepSeek R1 Zero – Since R1-Zero has some reasoning ability, researchers pick out reasoning results with good readability and reorganize them as cold-start data.
• Manual filtering and optimization – Humans review some data and refine the expression to make the reasoning process more intuitive and clear.

In the end, DeepSeek-R1 used thousands of cold-start data points for initial fine-tuning, followed by reinforcement learning training.

Reinforcement Learning

Other machine learning methods are mainly supervised learning and unsupervised learning. Reinforcement learning is the third paradigm, alongside these two.

The characteristics of reinforcement learning can be summarized in four points:

• No supervisor, only a reward signal
• Feedback is delayed, not immediate
• Has a time-series nature

• The agent's actions affect subsequent data

  Four Basic Elements

A reinforcement learning system generally includes four elements: policy, reward, value, and environment (or model). Let's introduce each of these four elements.

Policy A policy defines the actions an agent takes for a given state—in other words, a mapping from states to actions. In fact, states include both environmental states and agent states. We can summarize the characteristics of a policy in three points:

• A policy defines the agent's behavior
• It's a mapping from states to actions

• The policy itself can be a concrete mapping or a stochastic distribution

  Reward The reward signal defines the goal of the reinforcement learning problem. At each time step, the scalar value the environment sends to the reinforcement learner is the reward. We can summarize the characteristics of rewards in three points:

• A reward is a scalar feedback signal
• It indicates how well the agent performed at a given step
• The agent's task is to maximize the total accumulated reward over a period

Value Value, or the value function, is a very important concept in reinforcement learning. Unlike the immediacy of rewards, the value function measures long-term returns. We often say, "Keep your feet on the ground and your eyes on the stars." Evaluating the value function is like "looking at the stars"—judging the benefit of current actions from a long-term perspective, not just staring at immediate rewards. We can summarize the characteristics of the value function in three points:

• The value function is a prediction of future rewards
• It can assess the quality of a state
• Computing the value function requires analyzing transitions between states

Environment (Model)

The external environment, or model, is a simulation of the environment. Given a state and action, with a model we can predict the next state and corresponding reward. We can summarize the characteristics of the model in two points:

• The model can predict the environment's next step
• This prediction is reflected in the predicted state and reward

Reinforcement Learning Architecture

The Two Key Phases

Reinforcement Learning Phase One

Focuses on improving math and programming abilities. Starting from the cold-start checkpoint, it uses outcome-based reward-driven reinforcement learning scaling methods.

For math problems, the model uses a specialized accuracy verifier instead of a traditional reward model.

For programming tasks, a code execution server evaluates whether the code passes predefined test cases.

Reinforcement Learning Phase Two

Focuses on enhancing general capabilities.

The model is trained with a general reward model and rule-based verifiers.

Even with a small number of training steps, this significantly improves instruction following, human preference alignment, and agent performance. It also achieves general capability improvements without significantly degrading the math and programming abilities gained in Phase One.

Thanks to these techniques, QwQ can achieve a lot with a little—going toe-to-toe with large MoE models—and computing power has moved from the cloud to the edge (local). But why is there such a push to bring computing back to the edge, pursuing efficient models with low computing demands and running local computation?

Data Privacy in the AI Era

Every day, each of us leaves a massive amount of data on the internet.

And the vast majority of this data involves privacy concerns.

In the AI era, protecting personal privacy data has become a major challenge.

Apple has been focused on user data privacy for a very long time.

What they prioritize protecting is user data privacy—Huawei experts even wrote tens of thousands of words analyzing Apple's data privacy protection strategy.

Individual users can't have massive computing clusters locally. Right now, the only way for individual users to use large model features is to connect to the cloud, which poses a serious challenge to data privacy.

For example, some regions have banned Tesla cars out of concern that road data would be processed overseas. These are sensitive issues—we're deeply worried that AI-related data could be transmitted for illegal purposes or against national interests. So protecting data security is absolutely critical.

Running large models on-device can lower costs and improve privacy protection, which is why models need to be more efficient.

If a model is efficient enough, we can do more, and with higher precision, on-device with the same computing power.

One of the future directions of AI should be like QwQ: higher efficiency and lower on-device computing requirements.

Computing Power Demands in the AI Era

Thanks to advances in AI technology, we can now run AI models locally, even using CPU + memory for computation. The open-source project EXO distributes the layers of an AI model across different devices over the network, sharing computing power and further reducing AI operating costs.

But this only makes it possible to run—not to run efficiently or to train AI. For that, we still need extremely high computing power.

As mentioned earlier, all large models and their training methods are inseparable from "computing power." Whether it's the powerful DeepSeek or the efficient QwQ, neither can escape the need for computing power. Whether it's the 700GB VRAM requirement, a consumer-grade 4090 GPU, or even CPU-based computation, all of it relies on computing resources.

Global Electricity Demand Data Comparison

Global AI Computing Power Demand Data Comparison

Major Companies' Projected 2025 Investments

It's clear that with the explosive growth of AI, global demand for computing power has reached unprecedented levels, and electricity demand is also rising. Major players are racing to secure the high ground of computing power, highlighting just how crucial it is in the AI era.

At the same time, some have even started saying, "Computing power equals national power"!