Add Lab 4 inference settings visualization

This commit is contained in:
c4ch3c4d3
2026-04-27 14:50:55 -06:00
parent a7c1bda07c
commit 08c21fa0e2
6 changed files with 1348 additions and 5 deletions
+19 -4
View File
@@ -14,6 +14,7 @@ In this lab, we will:
- Run Open WebUI
- Using an Ollama Model within Open WebUI
- Visualizing Inference Parameters
- Experimenting with Inference Parameters
- Experimenting with Prompting Techniques
@@ -123,7 +124,19 @@ Locate, pull, and run **Qwen3.5 4B** using the **OpenWebUI**. By defualt, Ope
---
## Objective 3: Inference Settings
## Objective 3: Inference Settings Visualization
### Explore: Token Sampling Controls
Before changing model settings in Open WebUI, use these three toy samplers to see what the controls do to the next-token distribution. Each widget starts from the same prompt, `The quick brown fox`, and predicts candidate continuations toward the familiar phrase `jumps over the lazy dog`.
Temperature reshapes the whole distribution. Top K removes every candidate outside the K most likely tokens. Top P keeps the smallest group of candidates whose cumulative probability reaches P, while Min P keeps candidates above a probability floor relative to the strongest candidate.
<div data-inference-settings-visualization></div>
---
## Objective 4: Inference Settings
### Explore: OUI Inference Parameter Valves
@@ -205,7 +218,7 @@ Feel free to continue to explore with other topics or images. Note how each time
---
## Objective 4: Prompting Techniques
## Objective 5: Prompting Techniques
### Explore: Prompt Engineering & System Prompting
@@ -342,12 +355,14 @@ Throughout this lab, we've explored the fascinating world of Open WebUI and prom
- Top K: Limits token selection to top K most likely options
- Top P: Uses nucleus sampling based on cumulative probability
3. **Prompting Techniques**: We examined various prompting strategies:
3. **Inference Settings Visualization**: We used a local sampler to see how Temperature, Top K, Top P, and Min P reshape candidate token selection.
4. **Prompting Techniques**: We examined various prompting strategies:
- Few Shot Prompting: Providing examples of desired outputs
- Meta Prompting: Giving guidance to reach outcomes
- Chain of Thought: Encouraging step-by-step reasoning
- Self Criticism: Having the model evaluate its own responses
4. **System Prompting**: We created custom models with specific system prompts and parameter settings, learning how to tailor LLM behavior for specialized tasks.
5. **System Prompting**: We created custom models with specific system prompts and parameter settings, learning how to tailor LLM behavior for specialized tasks.
These concepts are foundational for effectively working with large language models in real-world applications. Remember that prompt engineering is both an art and a science - it requires understanding both the capabilities of the model and the nuances of human language. As you continue your journey with LLMs, don't hesitate to experiment with different approaches and parameters to find what works best for your specific use cases.