Building a local LLM on a Mac with mlx-serve

We installed mlx-serve and Qwen3-4B-4bit on a Mac, started the model locally and checked that its API answered. The times and speeds below were measured on November 5 and 6, 2026, on an Apple M1 Max with 64 GB of memory running macOS 26.6. We cannot promise the same numbers on another Mac.

What we used, and the licenses

We used mlx-serve-bin-macos-arm64.tar.gz from the v26.11.1 GitHub release. mlx-serve itself is MIT-licensed; the third-party code it bundles carries Apache-2.0 and BSD notices, which you can read in the NOTICE file in the archive. The model is mlx-community/Qwen3-4B-4bit, an Apache-2.0 release based on the original Qwen/Qwen3-4B. Before you use them, read the LICENSE and NOTICE in your own copy of the archive and the model's license. The project also documents installing from a DMG or with Homebrew; we did not try either here.

Check the archive and find the executable

We downloaded the archive from the official v26.11.1 GitHub release. Set ARCHIVE and UNPACK_DIR to where you actually saved the archive and want it unpacked. With both paths set, we ran these steps in zsh and in bash. If the SHA-256 does not match, nothing is unpacked; if the executable or its version cannot be confirmed, the next step will not run. The commands are exactly as we ran them, so the messages they print are in Japanese.

ARCHIVE="$HOME/Downloads/mlx-serve-bin-macos-arm64.tar.gz"
UNPACK_DIR="$HOME/Downloads/mlx-serve-v26.11.1"
if ! printf '%s  %s\n' '9acaa7a21b294536c3db9f2683d85079727b4e8db725d34260020499523dc267' "$ARCHIVE" | shasum -a 256 -c -; then
  printf '%s\n' 'SHA-256が一致しません。ここで中止してください。' >&2
elif ! mkdir -p "$UNPACK_DIR" || ! tar -xzf "$ARCHIVE" -C "$UNPACK_DIR"; then
  printf '%s\n' '解凍できませんでした。ここで中止してください。' >&2
else
  BIN="$(find "$UNPACK_DIR" -type f -name mlx-serve -print -quit)"
  if [ -z "$BIN" ] || [ ! -x "$BIN" ]; then
    unset BIN
    printf '%s\n' '実行可能な mlx-serve が見つかりません。' >&2
  elif ! "$BIN" --version; then
    unset BIN
    printf '%s\n' 'バージョンを確認できませんでした。' >&2
  fi
fi

The SHA-256 matched, and the first line of --version was mlx-serve 26.11.1 (fast). The archive was signed with team ID S77TW432YG (David Dalcu's Developer ID). The commands above do not check the signature. BIN now holds the path of the executable found in the unpacked folder. Run the next step in the same terminal.

Download the model and start the server

Downloading the model needs a network connection. The first download, on November 5, was 9 files totaling 2,173 MB and took about 54 seconds. The model is saved under ~/.mlx-serve/models/. On November 6 the model was already there, and pull carried on without downloading it again.

if [ -n "${BIN:-}" ] && [ -x "$BIN" ]; then
  "$BIN" pull mlx-community/Qwen3-4B-4bit &&
  "$BIN" --model "$HOME/.mlx-serve/models/mlx-community/Qwen3-4B-4bit" --serve --host 127.0.0.1 --port 11234
else
  printf '%s\n' '先にアーカイブとバージョンを確認してください。' >&2
fi

Within a few seconds of starting, the server printed Server listening on http://127.0.0.1:11234. The context length is set automatically from how much memory is free: on the same Mac it was 12,288 tokens at one time and 870 tokens at another.

Keep --host 127.0.0.1 in the command. The official cli.md gives the default as 0.0.0.0, while SECURITY.md says 127.0.0.1, so the two documents disagree. Here the server is not exposed to the local network.

Check the API responses

With the server still running, we ran the following in a second terminal, in this order, checking the HTTP status and the saved response each time. x-api-key: local-test is the value we used for local requests. The prompts are in Japanese, as we sent them; a translation follows each result.

OpenAI-compatible, without streaming

OUT_DIR="$(mktemp -d)"
curl -sS -o "$OUT_DIR/chat.json" -w 'HTTP %{http_code}\n' \
  -H 'Content-Type: application/json' \
  --data-binary @- http://127.0.0.1:11234/v1/chat/completions <<'JSON'
{"model":"mlx-community/Qwen3-4B-4bit","max_tokens":128,"messages":[{"role":"user","content":"ローカルLLMとは何か、3文で説明してください。"}]}
JSON
python3 -m json.tool --no-ensure-ascii "$OUT_DIR/chat.json"

On November 6 the response was HTTP 200. It generated 65 tokens at 81.0 tokens per second. The prompt asks for a three-sentence explanation of what a local LLM is, and the answer began "ローカルLLMとは、自社や組織のサーバー内に置かれる人工知能モデルのことです。" ("A local LLM is an artificial intelligence model kept on a company's or organization's own servers.")

OpenAI-compatible, streaming

curl -sSN -o "$OUT_DIR/stream.sse" -w 'HTTP %{http_code}\n' \
  -H 'Content-Type: application/json' \
  --data-binary @- http://127.0.0.1:11234/v1/chat/completions <<'JSON'
{"model":"mlx-community/Qwen3-4B-4bit","max_tokens":64,"stream":true,"messages":[{"role":"user","content":"日本の首都はどこですか。短く答えてください。"}]}
JSON
grep '^data:' "$OUT_DIR/stream.sse"

HTTP 200. The data: lines arrived in sequence, ending with data: [DONE]. The prompt asks for the capital of Japan in a short answer, and the answer was "東京" (Tokyo).

Anthropic-compatible, Messages API

curl -sS -o "$OUT_DIR/messages.json" -w 'HTTP %{http_code}\n' \
  -H 'Content-Type: application/json' \
  -H 'anthropic-version: 2023-06-01' \
  -H 'x-api-key: local-test' \
  --data-binary @- http://127.0.0.1:11234/v1/messages <<'JSON'
{"model":"mlx-community/Qwen3-4B-4bit","max_tokens":64,"messages":[{"role":"user","content":"推論を英語で一語で説明してください。"}]}
JSON
python3 -m json.tool --no-ensure-ascii "$OUT_DIR/messages.json"

HTTP 200. The prompt asks for the Japanese word 推論 to be explained in one English word. The answer was "Inference.", and stop_reason was end_turn. The generation speeds we observed on this Mac ranged from 34 to 90 tokens per second, depending on what else was running at the same time. They are not a measure of speed on any other model of Mac.

Errors, and what we have not tested

If the SHA-256 does not match, do not run that archive. On November 6, while another program was using GPU memory, the same three requests all returned HTTP 400. The error was "Prompt (29 tokens) requires ~661MB GPU memory but only ~53MB is available …". If you see it, first close the programs that are using the memory, then try again. For any other HTTP error, check the saved response and the server log.

We have not tested running with the network disconnected, the app, Homebrew or GGUF. The statement that there is no analytics or telemetry comes from the official FAQ; we did not measure the network traffic ourselves. To go further into evaluation, see how we check a LoRA adapter before it ships.