---
order: 2
title: "Lab 2 - Quantization Tradeoffs: Comparing 4-bit and 6-bit"
description: Compare Gemma 4 E2B in two Ollama quantizations and study how lower precision changes behavior.
---

<!-- breakout-style: instruction-rails -->
<!-- step-style: underline -->
<!-- objective-style: divider -->

In this lab, we will:

- Pull the same Gemma model in Q4 and Q6 Ollama variants
- Compare the quantization labels and model behavior across those variants
- Observe how lower precision changes the model's behavior
- Build intuition for when a smaller quant may or may not be worth it

<div class="lab-callout lab-callout--info">
  <strong>Lab Flow Guide</strong><br />
  <strong>Explore</strong> sections focus on comparison and trade-off analysis.<br />
  <strong>Execute</strong> sections require collecting evidence from each quantized model.
</div>

## Objective 1: Understand the Model and the Quantizations

For this lab, we will use two Ollama-published variants of **Gemma 4 E2B** that represent distinct precision bands:

| Precision band | Ollama model tag       | Why we are using it               |
| -------------- | ---------------------- | --------------------------------- |
| Q4             | `batiai/gemma4-e2b:q4` | Faster, smaller quant             |
| Q6             | `batiai/gemma4-e2b:q6` | Higher-quality quant in this lab  |

Even though the Ollama tags differ, these are all variants of the same underlying Gemma 4 E2B model family. The main variable we are changing is how the weights are stored.

When we say these files are the same model, we mean that the overall neural network is still the same:

- The same architecture
- The same layer count
- The same tokenizer
- The same training and instruction tuning
- The same general behavior the model learned during training

What changes is the numeric representation of the learned weights.

Imagine one learned weight in the original model is:

```text
0.156347
```

That number came from training. It is one of many values the model uses while computing each next token. Quantization does not invent a new model from scratch. Instead, it takes that trained value and asks:

```text
How can we store a close-enough version of this number using fewer bits?
```

If we use a simplified integer-style quantization scheme, the math looks like this:

```text
scale = max(|w|) / (2^(bits - 1) - 1)
q = round(w / scale)
w_hat = q * scale
```

Where:

- `w` is the original weight
- `q` is the stored integer bucket
- `scale` maps integers back into the original numeric range
- `w_hat` is the reconstructed approximation used at inference time

So if the original trained value was `0.156347`, a lower-bit quantized file may not store that exact number anymore. It may store an integer bucket like `1`, `5`, or `22`, plus a scale, and reconstruct an approximation such as:

- `0.000000`
- `0.130029`
- `0.146806`
- `0.157782`

Those are not identical to the original weight, but they may still be close enough for useful inference.

<div data-quantization-explorer></div>

### Explore: Interactive precision viewer

The viewer below zooms out from one weight and instead shows a toy layer with 16 stored values. Real GGUF schemes such as `Q4_K_M` and `Q6_K` are more sophisticated than this toy example, but the core idea is the same:

- Fewer bits means fewer representable values
- More weights get pushed into the same small set of stored buckets
- The layer becomes more compressed as precision drops

<div data-quantization-grid-explorer></div>

### Explore: Compare the same prompts through the hosted chat widget

By default, the widget below points to the courseware-managed Ollama service and the Lab 2 model tags above. You can still switch to another endpoint if your instructor provides one.

- Use the preloaded managed endpoint or replace it with another compatible endpoint
- Optionally add an API key if your chosen endpoint requires one
- Switch between the configured Q4 and Q6 Gemma variants
- Re-run the same prompt so you can compare coherence, stability, and SVG output
- Try a visual prompt such as `Draw a pelican riding a bicycle.`

The widget keeps the transcript in your browser so you can switch models without losing your place.  Refresh the page to clear the chat history.

<div data-objective5-chat></div>

## Objective 6: Reflect on the Tradeoff

By this point, you should have:

- Compared two quantized versions of the same model
- Measured the storage savings directly
- Verified that the core model metadata remains largely the same
- Observed where output quality begins to degrade

The important takeaway is not that one quant is always "best." The important takeaway is that quantization is a deployment decision. The right choice depends on your hardware limits, acceptable quality loss, and the task you need the model to perform.

## Conclusion

This lab isolates quantization as the main variable. By comparing **Gemma 4 E2B** in Q4 and Q6 Ollama variants, you can directly observe one of the most important tradeoffs in local inference: balancing model quality against efficiency and resource constraints.