Surrogate Model

Definition #

A Surrogate Model is a smaller, cheaper model trained to imitate a larger LLM for routine decisions.

Its role in MoM is:

Event → MoM → “This is routine” → Surrogate Model → Answer

So instead of using the expensive full LLM every time, MoM can send suitable tasks to the surrogate.

The Problem That Led to It #

Suppose our simulation produces 1 million routine decisions.

The full LLM can handle them, but using it for all 1 million decisions wastes compute.

We therefore want something that can say:

“For these common, simple situations, I can produce approximately the same decision as the large model, but much cheaper.”

That is the purpose of a surrogate model.

What Problem It Solves #

It reduces the amount of work sent to the expensive LLM.

The large LLM acts as the teacher, and the smaller model learns to imitate its outputs.

Teacher LLM → training → Student/Surrogate Model

What Happens If Not Used #

MoM would have fewer cheap alternatives.

Many routine events would still need to go to the full LLM, reducing the benefit of the MoM architecture.

Easy Wording #

A surrogate model is a smaller model that learned to handle the routine jobs normally handled by a much bigger model.

Layman Example #

Imagine an experienced doctor teaching a junior assistant.

The doctor handles difficult cases.

The assistant learns how the doctor handles common, simple cases and can handle those independently.

The doctor doesn't need to personally examine every simple case.

Technical Example #

Suppose the full LLM receives thousands of routine events.

During training:

Full LLM (Teacher)

↓

produces decisions for many examples

↓

Student Model learns those patterns

Later:

Event → MoM → routine? → Surrogate → decision

But:

Event → MoM → complex? → Full LLM → decision

So the surrogate doesn't replace the LLM completely. It handles the portion where imitation is good enough.

Limitation #

The surrogate is only an imitation.

For unusual or complex situations, it may produce a worse result than the teacher.

That's why MoM still needs the full LLM for difficult cases.

Solution #

Now the natural question is:

How does the small surrogate actually learn to imitate the large LLM?

That's exactly what Knowledge Distillation explains.