Add setup notes and Claude Code guidance for mac-mini LLM host
README: document VNC/screen-sharing and record the macOS 26 gotcha that the Screen Sharing authorization switch is GUI/MDM-only — launchctl/kickstart/SRP/access-group all fail. Add CLAUDE.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
28
CLAUDE.md
Normal file
28
CLAUDE.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## What this repo is
|
||||
|
||||
Operational notes (not code) for a remote LLM host: `mac-mini.internal.lotimmy.com` (Apple M4, 24 GB, arm64). `README.md` is the source of truth for hardware specs, installed tools, downloaded models, and the full command reference — read it before acting.
|
||||
|
||||
There is no build/lint/test — actual work runs on the remote host over SSH, so most "commands" are things to run *there*, not locally.
|
||||
|
||||
## Working on the remote host
|
||||
|
||||
- Connect by hostname only: `ssh mac-mini.internal.lotimmy.com`. Connecting by IP (`192.168.88.15`) fails host-key verification.
|
||||
- **Homebrew is not in the non-interactive SSH PATH.** Every remote command must load it first, or `brew`/`llmfit`/`llama-*` will be `command not found`:
|
||||
```bash
|
||||
ssh mac-mini.internal.lotimmy.com 'eval "$(/opt/homebrew/bin/brew shellenv)" && <command>'
|
||||
```
|
||||
|
||||
## Running llama.cpp (b8680 on this host)
|
||||
|
||||
- Single-shot inference → `llama-completion`. Interactive chat → `llama-cli`. **Do not pass `-no-cnv`** (unsupported in b8680; it prints an error and drops into chat anyway).
|
||||
- Gemma 4 models **require `--jinja`** or `llama-completion` throws `this custom template is not supported`.
|
||||
- Always pass `-ngl 999` to force all layers onto the GPU; auto-detection can silently fall back to CPU (~5× slower).
|
||||
- **Never pipe `llama-cli` output to `tail`** — interactive mode keeps printing `> ` prompts, so you see nothing and stdout balloons to GB. Use `llama-completion`, or redirect to a file and read it.
|
||||
|
||||
## When editing this repo
|
||||
|
||||
It's documentation. Keep the README's existing structure (踩雷紀錄 = gotchas log); append new gotchas there rather than scattering them.
|
||||
104
README.md
Normal file
104
README.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# mac-mini
|
||||
|
||||
家中 LLM 主機 `mac-mini.internal.lotimmy.com` 的設定與操作筆記。
|
||||
|
||||
## 硬體
|
||||
|
||||
- **機器:** Apple Mac mini, M4 (10 cores)
|
||||
- **記憶體:** 24 GB unified memory (Metal/MLX backend)
|
||||
- **架構:** arm64
|
||||
- **使用者:** `timmy`
|
||||
- **位址:** `mac-mini.internal.lotimmy.com` (`192.168.88.15`)
|
||||
|
||||
## 已安裝工具 (Homebrew, `/opt/homebrew`)
|
||||
|
||||
| 工具 | 版本 | 用途 |
|
||||
|---|---|---|
|
||||
| `llmfit` | 0.9.5 | 分析硬體適配的 LLM 模型、下載 GGUF |
|
||||
| `llama.cpp` | b8680 | LLM 推論引擎 (Metal backend 已驗證) |
|
||||
| `ggml` | 0.9.11 | llama.cpp 依賴 |
|
||||
|
||||
## 已下載模型
|
||||
|
||||
| 模型 | 路徑 | 大小 | 速度 |
|
||||
|---|---|---|---|
|
||||
| Gemma 4 E4B-it (Q8_0) | `~/.cache/llmfit/models/gemma-4-E4B-it-Q8_0.gguf` | 7.6 GB | ~18.7 tok/s (Metal) |
|
||||
|
||||
## 連線
|
||||
|
||||
```bash
|
||||
ssh mac-mini.internal.lotimmy.com
|
||||
```
|
||||
|
||||
> ⚠️ 用 IP (`192.168.88.15`) 連線會 host-key verification failed,固定用 hostname。
|
||||
|
||||
### VNC / 螢幕共享
|
||||
|
||||
本機開圖形桌面:
|
||||
|
||||
```bash
|
||||
open -n vnc://timmy@mac-mini.internal.lotimmy.com
|
||||
```
|
||||
|
||||
前提是遠端已在「系統設定 → 一般 → 共享 → 螢幕共享」把開關打開。**這個授權主開關只能用 GUI(或 MDM)開,SSH 完全翻不動** — 詳見踩雷紀錄第 6 條。
|
||||
|
||||
## 常用指令
|
||||
|
||||
### Homebrew / llmfit / llama.cpp
|
||||
|
||||
非互動 SSH session 載入 PATH:
|
||||
|
||||
```bash
|
||||
ssh mac-mini.internal.lotimmy.com 'eval "$(/opt/homebrew/bin/brew shellenv)" && <command>'
|
||||
```
|
||||
|
||||
### llmfit
|
||||
|
||||
```bash
|
||||
llmfit system # 顯示硬體規格
|
||||
llmfit fit --no-dashboard --limit 15 # 列出適配模型
|
||||
llmfit search <keyword> --no-dashboard # 搜尋模型
|
||||
llmfit info <model> --no-dashboard # 看單一模型詳情
|
||||
llmfit download <repo> --quant <quant> # 下載 GGUF
|
||||
```
|
||||
|
||||
### 跑 Gemma 4 E4B
|
||||
|
||||
```bash
|
||||
# 互動式對話
|
||||
llama-cli -m ~/.cache/llmfit/models/gemma-4-E4B-it-Q8_0.gguf -ngl 999 --jinja
|
||||
|
||||
# 單次推論 (非互動)
|
||||
llama-completion -m ~/.cache/llmfit/models/gemma-4-E4B-it-Q8_0.gguf \
|
||||
-p "你的提示" -n 200 -ngl 999 --jinja
|
||||
|
||||
# OpenAI 相容 API server (port 8080)
|
||||
llama-server -m ~/.cache/llmfit/models/gemma-4-E4B-it-Q8_0.gguf -ngl 999 --jinja
|
||||
```
|
||||
|
||||
## 踩雷紀錄
|
||||
|
||||
1. **brew 不在預設 SSH PATH** — 必須先跑 `eval "$(/opt/homebrew/bin/brew shellenv)"`,否則 brew/llmfit/llama-* 都會 `command not found`。
|
||||
|
||||
2. **`llama-cli -no-cnv` 在 b8680 不支援** — 會印一行錯誤訊息然後依然進入 chat 模式空轉。
|
||||
- 單次推論 → 用 `llama-completion`
|
||||
- 互動聊天 → 用 `llama-cli` (不加 `-no-cnv`)
|
||||
|
||||
3. **Gemma 4 必須加 `--jinja`** — 否則 `llama-completion` 會丟 `std::runtime_error: this custom template is not supported, try using --jinja`。
|
||||
|
||||
4. **`-ngl 999` 強制全部層丟 GPU** — 沒指定時 auto-detection 可能掉到 CPU,慢約 5 倍。
|
||||
|
||||
5. **驗證輸出時不要 pipe `tail`** — `llama-cli` 互動模式會持續印 `> ` prompt,pipe `tail` 看不到生成內容,還會把 stdout 撐到 GB 級。要嘛用 `llama-completion`,要嘛 redirect 到檔案再讀。
|
||||
|
||||
6. **macOS 26 開螢幕共享:只能 GUI/MDM,別浪費時間在 CLI** — 「連線失敗 / 不允許螢幕共享,請在系統設定停用並重新啟用」這個錯誤,**不是** launchctl / kickstart / SRP / 存取群組能修的。實測全部無效:
|
||||
- `launchctl enable/bootstrap system/com.apple.screensharing` → daemon 有跑、port 5900 有 LISTEN,但握手仍被拒
|
||||
- `kickstart -activate`(啟用 Remote Management)→ 印警告 `Screen recording/control might be disabled ... must be enabled from System Settings or via MDM`,本身就是官方在說 CLI 開不動
|
||||
- `AllowSRPForNetworkNodes -bool true`、加 `com.apple.access_screensharing` 群組 → 都沒用
|
||||
- **唯一解:實體到那台**「系統設定 → 一般 → 共享 → 螢幕共享」把開關打開(已開就關掉再開,對應錯誤訊息的「停用並重新啟用」)。這個授權主開關受保護,SSH 摸不到。→ headless 機器至少要接一次螢幕鍵鼠開好它。
|
||||
- `sudo ssh` 要密碼但 `!` 前綴管道無 tty → sudo 會 `a terminal is required`;這類指令要在真正的 Terminal.app 跑。
|
||||
|
||||
## 24 GB 記憶體下的模型選擇
|
||||
|
||||
- 旗艦 (吃緊): `gemma-4-26B-A4B-it` MoE → 16.8 GB at 4-bit (70% 記憶體)
|
||||
- 平衡: `gemma-4-E4B-it` Q8 → 9 GB (37% 記憶體) ← 目前使用
|
||||
- 同類選擇: `Qwen3-Coder-30B-A3B-Instruct` MoE 系列 (llmfit 評分前段)
|
||||
Reference in New Issue
Block a user