66 lines
1.8 KiB
Markdown
66 lines
1.8 KiB
Markdown
|
# Hardware envelope — homelab accessible (draft)
|
|||
|
|
|
|||
|
|
**Status:** assumptions + TBD measurements
|
|||
|
|
**Used by:** inventory `hardware_class`, axis B/C prioritization, collection size decisions
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Purpose
|
|||
|
|
|
|||
|
|
Define what Freedom Intelligence means by **homelab accessible** so we do not
|
|||
|
|
collect models that only make sense on hyperscale clusters.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Working tiers (until hosts are measured)
|
|||
|
|
|
|||
|
|
| Tier | Assumed capacity | Target model class |
|
|||
|
|
| ---- | ---------------- | ------------------ |
|
|||
|
|
| **T0 — CPU / edge** | 16–64 GB system RAM, no GPU | ≤3B Q4; embeddings ≤0.5B |
|
|||
|
|
| **T1 — consumer GPU** | 8–12 GB VRAM | 7–8B Q4/Q5; 3B fp16 |
|
|||
|
|
| **T2 — enthusiast GPU** | 16–24 GB VRAM | 14B Q4; 8B fp16; light 32B Q4 |
|
|||
|
|
| **T3 — lab multi-GPU** | 2×24 GB+ or 48 GB+ | 32B fp16 / 70B Q4; small MoE |
|
|||
|
|
| **T4 — out of envelope** | multi-node / 8×A100-class | Full DeepSeek-V3 MoE — P2 watch only |
|
|||
|
|
|
|||
|
|
P0 collection targets **T0–T2**. P2 full MoE is **T4**.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Lab hosts (fill in)
|
|||
|
|
|
|||
|
|
| Host | Role | GPU | VRAM | RAM | Notes |
|
|||
|
|
| ---- | ---- | --- | ---- | --- | ----- |
|
|||
|
|
| TBD | primary local inference | TBD | TBD | TBD | |
|
|||
|
|
| TBD | training experiments | TBD | TBD | TBD | |
|
|||
|
|
| railiance01 | cluster (not weight store) | n/a | n/a | n/a | Prefer not to fill hot disks with weights |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Inference runtime defaults (intent)
|
|||
|
|
|
|||
|
|
| Runtime | When |
|
|||
|
|
| ------- | ---- |
|
|||
|
|
| llama.cpp / GGUF | Desktop & edge |
|
|||
|
|
| vLLM | Server multi-request |
|
|||
|
|
| MLX | Apple silicon if present |
|
|||
|
|
| Ollama | Quick operator UX only |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Training defaults (intent)
|
|||
|
|
|
|||
|
|
| Method | Envelope |
|
|||
|
|
| ------ | -------- |
|
|||
|
|
| QLoRA 7–8B | T1–T2 |
|
|||
|
|
| QLoRA 14B | T2 |
|
|||
|
|
| Full FT 7B | T2–T3 |
|
|||
|
|
| 70B+ FT | T3+ only with explicit plan |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Pin log
|
|||
|
|
|
|||
|
|
| Date | Change |
|
|||
|
|
| ---- | ------ |
|
|||
|
|
| 2026-07-24 | Draft tiers; host table empty |
|