Skip to content

fw infer

Send a prompt to an OpenAI-compatible endpoint; with –gguf, read the GGUF header and run a Q8_0 dot-product sample (no token generation)

Terminal window
fw infer [prompt] [--gguf <path>] [--model <m>] [--url <u>]

fw run runs fw infer.

Flag Argument
--gguf <path>
--model <m>
--url <u>

Every command also accepts -C, --directory, --offline, --json, -q and -v. See the overview.

fw train, fw hf, fw abliterate, fw vram, fw heretic, fw align

Read from fw infer --help of fw 2026.11.0.