plamo-translate-cli

A command-line interface for translation using the plamo-2-translate model with local execution.

Features

• Translate text between 16+ languages including Japanese, English, Chinese, Korean, and more
• Simple command-line interface for easy integration into scripts and workflows
• MLX and native llama.cpp backends, including Metal acceleration on Apple Silicon

Installation

For macOS

plamo-translate currently installs on Python 3.10 through 3.14 on macOS. No additional workaround is required for sentencepiece on Python 3.13 or 3.14 with current upstream releases.

pip install plamo-translate

uv tool

If you use uv as a package manager rather than pip, you can install plamo-translate into an isolated environment:

uv tool install -p 3.14 plamo-translate

Development

uv sync
source .venv/bin/activate

Requirements

• Python 3.10 through 3.14
  • Common dependencies:
    • mcp[cli]
    • numba
  • On macOS:
    • mlx-lm

Usage

Basic usage

You can specify the input and output language by giving --from and --to options. If you don't specify them, the input/output language will be automatically selected from English or Japanese.

Interactive mode

$ plamo-translate
Loading models...done!
Interactive mode enabled. Type your input below (Ctrl+D to exit).
> こんにちは、お元気ですか？
Hello, how are you?
> 「お腹減った〜何食べたい？」「私はうなぎ！」
"I'm hungry! What do you want to eat?" "I want eel!"
> You translate ambiguous expression in Japanese into English very well.
あなたは日本語の曖昧な表現を英語に翻訳するのがとても上手です。

Pipe mode

$ cat file.txt | plamo-translate
The virtual worlds of the internet have experienced remarkable technological advancement. Meanwhile, the real world still contains numerous areas where technology has yet to make significant inroads, with many inefficient manual tasks and dangerous work still requiring human intervention. This situation stems from the fact that conventional technology has struggled to adapt to the dynamic changes and diverse conditions of the real world.

PFN's core strengths lie in machine learning and deep learning technologies, which demonstrate exceptional flexibility in handling uncertainty and have the potential to create significant impact in the real world. For example, by applying deep learning technologies to robots that excel at repetitive tasks, we can enable them to make more human-like flexible judgments and perform complex tasks.

To create meaningful impact in the real world, it's essential to push the boundaries of cutting-edge technology and research application domains where technological innovation can create tangible change. For these purposes, PFN assembles a team of exceptionally talented professionals with diverse expertise.

Server mode

First, launch the server:

$ plamo-translate server

Then, use the client mode:

$ plamo-translate --input '家計は火の車だ'
Our household is in financial trouble.

You can also use the interactive mode with the server:

$ plamo-translate
Loading models...done!
Interactive mode enabled. Type your input below (Ctrl+D to exit).
> 家計は火の車だ
Our household is in financial trouble.

It can skip the loading time of the model, so it is useful when you want to use this tool frequently.

Using from MCP Client

The plamo-translate server command starts an MCP (Model Context Protocol) server. This allows plamo-translate to be used as a tool in other applications that support MCP, such as Claude Desktop.

Here, we introduce how to use plamo-translate with Claude Desktop, which is a popular MCP client.

1. Start the plamo-translate server:

   plamo-translate server
   

2. In a new terminal, run the following command to display the MCP configuration for Claude Desktop:

   plamo-translate show-claude-config
   

   and you will see the configuration in JSON format as follows:

   {
     "mcpServers": {
       "plamo-translate": {
         "command": "/path/to/npx",
         "args": [
           "-y",
           "mcp-remote",
           "http://localhost:8000/mcp",
           "--allow-http",
           "--transport",
           "http-only"
         ],
         "env": {
           "PATH": "[THE SAME STRING AS YOUR CURRENT PATH ENVIRONMENT VARIABLE]",
         }
       }
     }
   }
   

3. Copy the outputted configuration.
4. Paste this configuration into your Claude Desktop's MCP configuration file (on macOS, this is typically located at ~/Library/Application Support/Claude/claude_desktop_config.json).

Once configured, you can use plamo-translate directly from Claude Desktop.

Select precision of the model weight

You can specify the precision of the model weight by giving a --precision option.

$ plamo-translate server --precision 8bit

Use original local weights with the optimized MLX backend

uv run plamo-translate server \
  --model ~/Models/pfnet--plamo-2-translate --precision 4bit
uv run plamo-translate --from English --to Japanese < tests/fixtures/translation.en.txt

--model accepts a local checkpoint directory or a Hugging Face repository and takes precedence over PLAMO_TRANSLATE_CLI_MODEL_NAME. Original weights are quantized once during loading; the source files are never modified. By default, original weights use 4bit and exported checkpoints retain their declared precision, including unquantized BF16 exports. Request --precision bf16 to retain the original weights. Changing a quantized checkpoint's precision requires loading the original weights again.

For faster subsequent startup, save a separate copy first:

uv run python scripts/prepare_mlx_model.py \
  --model ~/Models/pfnet--plamo-2-translate \
  --output ~/Models/pfnet--plamo-2-translate-mlx-4bit \
  --precision 4bit
uv run plamo-translate server --model ~/Models/pfnet--plamo-2-translate-mlx-4bit

mixed uses 4bit MLPs, 8bit attention/large Mamba projections and embeddings, and original precision for the small Mamba state projections. 4bit and 6bit are also available. These settings change model numerics and may change translations. Stop an existing server before changing its model or precision.

The backend honors the checkpoint's local/full-attention RoPE bases and PLaMo's unclipped SSM time steps, keeps recurrent state in FP32, and preserves the original BF16 activation dtype. An experimental fused decode convolution is available with PLAMO_TRANSLATE_CLI_OPTIMIZE=1; it is disabled by default because measured speed was indistinguishable from the upstream convolution and rounding changed some tokens. The prompt includes the checkpoint's BOS token. GPU requests are serialized; streaming also returns a final complete result to recover missing progress messages.

Reproduce the translation benchmark

uv sync --locked
uv run python scripts/benchmark_mlx.py \
  --model ~/Models/pfnet--plamo-2-translate \
  --precision 4bit --runs 3 --output .local/benchmarks/my-4bit

The supplied full English/Japanese example is in tests/fixtures/translation.en.txt and tests/fixtures/translation.ja.txt. Each process runs a short warmup, then uses greedy decoding with a fresh cache for every measured run, and records the full translation, token IDs, EOS/length termination, load time, prompt/decode throughput, memory, package versions, backend hash. A host lock prevents concurrent runs of this script; stop other GPU workloads while measuring. Timing excludes loading and warmup.

Quality checks include chrF against the supplied reference (higher is better) and teacher-forced reference negative log likelihood (NLL, lower is better). They measure this one example, not general translation accuracy; also inspect omissions, names, paragraphs and repetition in the saved output. --implementation corrected (the default) uses the production backend; optimized enables the experimental convolution; stock --legacy-prompt reproduces the upstream model and old CLI prompt. The benchmark also supports --dtype float16 as an explicit numerical experiment.

Measured result (2026-10-07)

Apple M1 Max, 32 GPU cores, 64 GB; MLX 0.31.1 / mlx-lm 0.31.2, Python 3.13.15. Source checkpoint revision: cae8da342a3e051ed69f90ce24c23eacff908732. Two warm runs per setting, batch size 1, temperature 0, prefill chunk 512, full supplied example. Other llama.cpp workloads were stopped for the final comparison.

The selected 4bit mode retains the old 4bit speed while improving this example's quality. Quantization accounts for most of the speed advantage over original BF16 weights. BF16 timing varied (13.45–17.32 tok/s); selected 4bit runs were 51.41–51.46 tok/s. These are local measurements, not a claim of globally optimal speed or unchanged accuracy on other documents/languages. The old-model comparison uses the same original checkpoint quantized locally, not a different community revision.

The selected output has all eight paragraphs, the title's “Together with You”, the partner/social-implementation paragraph, both founders, and “Learn or Die”. The old output dropped the title and the separate partner paragraph. The selected output is not identical to the reference: it still includes awkward wording such as 真に価値ある価値, and uses あなたと共に rather than the reference's 皆様と共に. The original BF16 output also omitted the title's “Together with You”. One reference cannot establish general quality; use 8bit/BF16 when validating other material if small quantization changes are unacceptable.

The experimental fused convolution was not selected: 51.66 versus 51.44 tok/s is too small a difference to establish a win, and its output changed. Converting all floating weights/activations to FP16 was rejected: reference logits became nonfinite, the output repeated the unknown token and reached the 2048-token limit. No FP16 conversion is applied by the production loader.

Only reviewed summaries are versioned under benchmarks/. Raw measurements, translations and logs stay in ignored .local/ directories. The benchmark does not collect a machine-wide process list. The saved 4bit checkpoint is 5.36 GB. Reloading that checkpoint through the MCP server took 4.68 seconds; complete CLI translations took 10.64 seconds (streaming) and 10.23 seconds (non-streaming), and both matched direct inference byte-for-byte. The selected weights are published at mlx-community/plamo-2-translate. That revision includes the corrected standalone MLX implementation and BOS/EOS settings. Its output matched the selected translation exactly, and all 15 published files were verified by SHA-256. This release uses ordinary affine quantization; it replaces the previous DWQ-labeled release, whose quality was not compared here. Reproduce this check with:

uv run python scripts/validate_mlx_cli.py \
  --model ~/Models/pfnet--plamo-2-translate-mlx-4bit \
  --expected .local/benchmarks/my-4bit/translation-0.txt \
  --output .local/benchmarks/my-cli-roundtrip

Reference implementations: PFN checkpoint and upstream MLX PLaMo 2.

Standalone 8bit and BF16 releases

Both releases reproduce corrected direct inference at the same precision in two standalone runs and both CLI modes. chrF uses the supplied example only; BF16 still omits “Together with You” in the title. All published files match the validated local bytes. A summary of the measurements and publication checks is in benchmarks/releases/2026-10-07.

--precision 8bit selects mlx-community/plamo-2-translate-8bit and --precision bf16 selects mlx-community/plamo-2-translate-bf16 unless a model path or repository was explicitly provided. The exporter supports both precisions:

uv run python scripts/prepare_mlx_model.py \
  --model /path/to/pfnet-plamo-2-translate --output /path/to/mlx-8bit --precision 8bit
uv run python scripts/prepare_mlx_model.py \
  --model /path/to/pfnet-plamo-2-translate --output /path/to/mlx-bf16 --precision bf16

Each export includes the standalone modeling_mlx_plamo2.py, a BOS-aware chat template, and both EOS IDs. BF16 export preserves the source floating weights without quantization; 8bit uses affine group-64 quantization while retaining BF16 floating parameters. No FP16 activation conversion or experimental convolution is enabled. A saved BF16 export also retains its precision when loaded via --model without --precision.

Validate against the full output of corrected direct inference at the same precision, then validate both CLI delivery modes:

uv run python scripts/validate_mlx_release.py \
  --model /path/to/mlx-8bit --precision 8bit \
  --source-revision cae8da342a3e051ed69f90ce24c23eacff908732 \
  --expected /path/to/direct-8bit-translation.txt --output /path/to/validation-8bit
uv run python scripts/validate_mlx_cli.py \
  --model /path/to/mlx-8bit --expected-precision 8bit \
  --expected /path/to/direct-8bit-translation.txt --output /path/to/cli-8bit

Use bf16 and the corresponding source-precision output for BF16 validation. The standalone check calls mlx_lm.load directly, verifies identical structured and natural-message prompts, requires EOS, compares the complete output, and hashes every inference asset. Equality is with the same-precision implementation; it does not imply equality between 8bit and BF16 or with the supplied human reference.

For publication, prepare the model card and license files, then run the local preflight below. Add --upload to publish using existing Hugging Face credentials:

uv run python scripts/publish_mlx_model.py \
  --model /path/to/mlx-8bit --repo mlx-community/plamo-2-translate-8bit \
  --validation /path/to/validation-8bit/validation.json \
  --expected-head PREVIOUS_HUGGING_FACE_COMMIT --output /path/to/publication-8bit

Publication rejects modified inference files, guards against concurrent remote updates, removes obsolete weight shards in the same commit, and verifies remote SHA-256 values after upload. It uses an explicit file allowlist; local provenance paths and supplied translation text are excluded from the model repository.

Supported Languages

• Japanese
• Japanese(easy)
• English

Experimentally Supported Languages

• Chinese
• Taiwanese
• Korean
• Arabic
• Italian
• Indonesian
• Dutch
• Spanish
• Thai
• German
• French
• Vietnamese
• Russian

Server Backends

• mlx: Optimized for macOS with Apple Silicon (default on macOS)
• llama.cpp: Local GGUF inference with the native llama-server, exposed through the same CLI and MCP tool

llama.cpp on Apple Silicon

Build the tested runtime from this checkout (requires Git, CMake and Xcode Command Line Tools):

bash scripts/build_llama_cpp.sh

The build pins llama.cpp b11318 (db33d3cb89d8b0d954df66047b13e9cc22df8f10) and applies scripts/patches/llama-cpp-plamo2-rope.patch. This fixes a PLaMo 2 loader bug: when layer 0 is Mamba, the common loader sets the rotary dimension to zero, silently disabling RoPE. The patch restores the attention head dimension. An unpatched runtime can produce fluent translations with missing paragraphs even with F16 weights. The backend rejects a runtime that reports n_rot = 0. The pinned revision also includes the PLaMo BOS/EOS tokenizer fix.

Convert the original local weights once; the original model directory is read only:

uv venv .local/convert-venv
uv pip install --python .local/convert-venv/bin/python \
  'torch==2.14.1' 'transformers==4.57.6' 'huggingface-hub<1' \
  sentencepiece numpy safetensors protobuf -e .local/llama.cpp/gguf-py
uv run python scripts/prepare_llama_cpp.py \
  --model-dir ~/Models/pfnet--plamo-2-translate \
  --llama-cpp-dir .local/llama.cpp \
  --python .local/convert-venv/bin/python \
  --output-dir ~/Models/pfnet--plamo-2-translate-gguf \
  --quantization Q4_0-ssm-f16

Conversion saves F16 plus the requested quantizations, logs, checksums, source revision and a copy of the model license. It refuses to overwrite existing weights. F16 avoids low-bit weight quantization; activation and normalization arithmetic differ from MLX, so numerical equivalence is not assumed. Allow about 26 GB for F16 plus the recommended GGUF, in addition to the source.

The recommended Q4_0-ssm-f16 recipe uses Q4_0 with all Mamba ssm_out matrices kept in F16. On the tested M1 Max, this preserves content omitted by plain Q4_0 while keeping most of its speed. See the measured comparison and quality summary. Other available recipes are Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q5_0, Q4_0 and Q4_1.

Start a persistent server, then translate using the existing client:

uv run plamo-translate server --backend-type llama.cpp \
  --llama-server .local/llama.cpp/build/bin/llama-server \
  --model ~/Models/pfnet--plamo-2-translate-gguf/plamo-2-translate-Q4_0-ssm-f16.gguf

# In another terminal:
uv run plamo-translate --from English --to Japanese < tests/fixtures/translation.en.txt

For one-shot use, omit server and pass the backend, executable and model options with the input. --no-stream, interactive translation, and show-claude-config work with either backend. A plain client reuses the running backend. Explicitly requesting a different model/backend fails without changing the running server's configuration. Use a separate TMPDIR to run both backends. The native process binds to loopback and is terminated when its owning MCP server exits.

--precision selects MLX weights; for llama.cpp, choose the GGUF file with --model. PLAMO_TRANSLATE_CLI_MODEL_NAME can also specify the GGUF path, and PLAMO_TRANSLATE_CLI_LLAMA_SERVER can specify the executable.

The llama.cpp defaults are full GPU offload (--gpu-layers 99), Flash Attention, F16 KV cache, one inference slot, a 32768-token context, 2048-token batches, 512-token microbatches, and 8 CPU threads. Override them with --ctx-size, --batch-size, --ubatch-size, --threads, --gpu-layers, and --flash-attn on|off|auto. Use --gpu-layers 0 for CPU execution. Greedy decoding, disabled repetition penalty, the translation-specific prompt, and the model's BOS/EOS settings preserve the intended translation behavior. Prompt caching is disabled for repeatable independent requests. Token/context exhaustion and interrupted responses are reported as errors, including in streaming mode.

Reproduce the supplied English/Japanese quality and speed comparison:

uv run --with sacrebleu python scripts/benchmark_llama_cpp.py \
  --llama-server .local/llama.cpp/build/bin/llama-server \
  --models ~/Models/pfnet--plamo-2-translate-gguf/plamo-2-translate-{F16,Q4_0-ssm-f16}.gguf \
  --repeats 2 --output .local/benchmark-new-run

The benchmark saves full translations, raw streaming events, native logs, time to first token, prompt/decode timings, and chrF against the supplied reference. It warms model weights with a separate short prompt and measures complete translations without reusing the prompt cache. Keep other GPU inference, quantization and media workloads idle while measuring. chrF is reference overlap for this single example; review the actual translations for missing content and do not interpret it as general translation accuracy.

Options

• --input TEXT Input text to translate
• --from TEXT Input language for translation (default: English)
• --to TEXT Output language for translation (default: Japanese)
• --model Local model directory or Hugging Face repository
• --precision Model weight precision: [4bit, 6bit, 8bit, mixed, bf16] (default: 4bit for original weights)

Configuration

You can configure the following parameters using environment variables:

• PLAMO_TRANSLATE_CLI_SERVER_START_PORT: Specifies the starting port number for the server.
• PLAMO_TRANSLATE_CLI_SERVER_END_PORT: Specifies the ending port number for the server.
• PLAMO_TRANSLATE_CLI_MODEL_NAME: Local model directory or Hugging Face repository.
• PLAMO_TRANSLATE_CLI_PRECISION: Explicit weight precision, equivalent to --precision.
• PLAMO_TRANSLATE_CLI_OPTIMIZE: Set to 1 to opt in to the experimental decode convolution (default: 0; correctness fixes are always enabled).
• PLAMO_TRANSLATE_CLI_PREFILL_STEP_SIZE: Prompt chunk size (default: 512).
• PLAMO_MAX_TOKENS: Maximum generated tokens (default: 32768). Reaching this limit reports an incomplete translation error.
• PLAMO_TRANSLATE_CLI_TEMP: Sets the temperature for text generation.
• PLAMO_TRANSLATE_CLI_TOP_P: Sets the top-p (nucleus) sampling probability.
• PLAMO_TRANSLATE_CLI_TOP_K: Sets the top-k sampling number.
• PLAMO_TRANSLATE_CLI_REPETITION_PENALTY: Sets the repetition penalty.
• PLAMO_TRANSLATE_CLI_REPETITION_CONTEXT_SIZE: Sets the context size for repetition penalty.

Deploy

bash scripts/deploy.sh