Filter by type:
Filter by tags:
Attention: Trust Remote Code is required for this model
wemm-embedding-2b
WeMM-Embedding-2B is Tencent's Apache-2.0 multilingual embedding model
built on Qwen3.5. This entry serves the original bfloat16 safetensors with
LocalAI's Transformers backend and produces 2,048-dimensional normalized
embeddings for text retrieval, semantic search, and RAG.
The upstream model can also embed images and videos. LocalAI currently
exposes text input through its embeddings API for this backend.
Repository: localaiLicense: apache-2.0
Attention: Trust Remote Code is required for this model
wemm-embedding-4b
WeMM-Embedding-4B is Tencent's mid-sized Apache-2.0 multilingual embedding
model built on Qwen3.5. This entry serves the original bfloat16 safetensors
with LocalAI's Transformers backend and produces 2,560-dimensional
normalized embeddings for text retrieval, semantic search, and RAG.
The upstream model can also embed images and videos. LocalAI currently
exposes text input through its embeddings API for this backend.
Repository: localaiLicense: apache-2.0
Attention: Trust Remote Code is required for this model
wemm-embedding-9b
WeMM-Embedding-9B is Tencent's largest Apache-2.0 multilingual embedding
model built on Qwen3.5. This entry serves the original bfloat16 safetensors
with LocalAI's Transformers backend and produces 4,096-dimensional
normalized embeddings for text retrieval, semantic search, and RAG.
The upstream model can also embed images and videos. LocalAI currently
exposes text input through its embeddings API for this backend.
Repository: localaiLicense: apache-2.0
dfm-mimir:vllm
DFM Mimir is an Apache-2.0, instruction-tuned HRM-Text model from Danish
Foundation Models. It has about 1 billion parameters and a 4,096-token
context window. The model focuses on Danish and English chat, reasoning,
mathematics, and code generation, and uses only permissible post-training
data. This entry serves the official BF16 safetensors checkpoint with vLLM.
Repository: localaiLicense: apache-2.0
carnice-v2-27b
# Carnice-V2-27B for Hermes Agent
Carnice-V2-27B is a full merged BF16 SFT of `Qwen/Qwen3.6-27B` for Hermes-style agent traces. This repository contains the standalone merged model weights, not only a LoRA adapter.
## BF16 Transformers Loading Fix
The BF16 safetensors were republished with corrected `Qwen3_5ForConditionalGeneration` tensor prefixes. The original merge artifact accidentally serialized an extra Unsloth wrapper prefix, which caused direct HF Transformers loads to report the real weights as unexpected keys and initialize expected layers randomly. GGUF files were not affected because the GGUF conversion path normalized those prefixes.
## Benchmarks
The benchmark artifact bundle is included under `benchmarks/`. It contains the rendered graph, extracted `metrics.json`, benchmark scripts, and raw result files used to make the chart.
Scope note: the IFEval run is a short `limit=20` A/B smoke benchmark, not an official full leaderboard score. Held-out loss/perplexity is the exact assistant-only training-format validation metric from the SFT script. The raw BFCL two-case smoke files are included for auditability, but they are too small to use as a model-quality claim.
...
Repository: localaiLicense: apache-2.0
qwen3.8-27b-exl3-vllm-cpp
Qwen3.8-27B EXL3 3.5bpw served by vllm.cpp, LocalAI's C++ vLLM-style
runtime. The published checkpoint generates on CUDA and measured 16.7
tokens/s on GB10 with the pinned revision and the limits configured here.
This is the target-only setup. Use the DFlash2 variant for the measured
speculative-decoding configuration. The entry downloads the complete
revision-pinned repository, including its configuration, tokenizer, index,
and safetensors shards.
Repository: localai
granite-4.2-3b:vllm
Granite 4.2 3B is IBM's compact dense reasoning model for code generation,
tool calling, agentic workflows, multilingual chat, and long-context tasks.
This entry serves the bfloat16 safetensors with vLLM and supports a
128K-token context. It is the smallest fallback in a family that also offers
the higher-capacity 8B and 30B checkpoints as variants.
Repository: localaiLicense: apache-2.0
granite-4.2-8b:vllm
Granite 4.2 8B is IBM's mid-sized dense reasoning model for code generation,
tool calling, agentic workflows, multilingual chat, and long-context tasks.
This entry serves the higher-capacity bfloat16 safetensors with vLLM and
supports a 128K-token context.
Repository: localaiLicense: apache-2.0
granite-4.2-30b:vllm
Granite 4.2 30B is IBM's largest dense Granite 4.2 reasoning model for code
generation, tool calling, agentic workflows, multilingual chat, and
long-context tasks. This entry serves the bfloat16 safetensors with vLLM and
supports a 128K-token context.
Repository: localaiLicense: apache-2.0
Attention: Trust Remote Code is required for this model
phonellm-alpha-1:vllm
PhoneLLM Alpha 1 is a 30B-parameter, 3.5B-active mixture-of-experts model
tuned by the Pipecat team for low-latency voice agents, multi-turn
conversations, and accurate tool use without thinking. This entry serves
the bfloat16 safetensors with vLLM and supports a 262K-token context.
PhoneLLM is released under BSD-2-Clause and derives from NVIDIA Nemotron 3
Nano 30B-A3B. The underlying work remains subject to the NVIDIA Nemotron
Open Model License and its redistribution and attribution requirements.
Repository: localaiLicense: bsd-2-clause
Attention: Trust Remote Code is required for this model
phonellm-alpha-1:sglang
PhoneLLM Alpha 1 is a 30B-parameter, 3.5B-active mixture-of-experts model
tuned by the Pipecat team for low-latency voice agents, multi-turn
conversations, and accurate tool use without thinking. This entry serves
the bfloat16 safetensors with SGLang and supports a 262K-token context.
PhoneLLM is released under BSD-2-Clause and derives from NVIDIA Nemotron 3
Nano 30B-A3B. The underlying work remains subject to the NVIDIA Nemotron
Open Model License and its redistribution and attribution requirements.
Repository: localaiLicense: bsd-2-clause
ltx-2.3-22b-dev-ggml
LTX-2.3 22B dev - DiT-based audio-video foundation model from Lightricks,
GGUF-quantized for the stable-diffusion.cpp backend. Generates synchronized
video and audio from a text prompt (T2V), a reference image (I2V), or
first/last frame pairs (FLF2V). Uses gemma-3-12b-it as the text encoder
and ships dedicated video and audio VAEs plus an embeddings_connectors
safetensors that bridges the LLM hidden states to the diffusion model.
This entry uses the dynamic (UD) Q4_K_M quantization of the 22B model
(~16 GB) paired with the UD-Q4_K_XL QAT Gemma encoder (~7.4 GB).
Recommended generation: width=1280, height=720, video_frames=33,
fps=24, sampler=euler, cfg_scale=6.0.
Repository: localaiLicense: ltx-2-community-license-agreement
kev-0.8b-vllm-cpp
kev is a System 1 decision model by Jared Palmer. It answers typed choice,
noul and score questions about a text state with one scoring pass per
question. It does not generate text. The model is a frozen
Qwen3.5-0.8B-Base backbone, a rank-16 LoRA adapter and a PointerHead
readout.
This entry installs a converted redistribution of jaredpalmer/kev-0.8b:
the LoRA is merged into the BF16 backbone, the head is stored as
head.safetensors, and config.json names the KevModel architecture. The
checkpoint only works with vllm.cpp (the vllm-cpp backend); transformers,
vLLM and llama.cpp cannot load it.
In LocalAI, serve it via POST /v1/systemone. The vllm.cpp project records
PointerHead golden-vector tests (25 cases) and a 5-case end-to-end
comparison against the kev reference server as equal. The upload itself
was smoke-tested with one request on CPU; there is no accuracy benchmark
and no GPU run. The entry sets a 2048-token context and an explicit KV
pool, because the default 4096-token context does not fit the default
CPU KV pool and the load fails. BF16 weights, about 1.53 GB, pinned to a
revision.
Repository: localaiLicense: apache-2.0
clm-v0.1-8b-vllm-cpp
CLM is a bi-encoder decision model from Contrastive-LM. A frozen Qwen3-8B
backbone encodes the state and each candidate answer separately, two MLP
heads project them into a 512-dimensional space, and the answer
distribution is a softmax over the cosine similarities at scale 100. It
answers typed choice, noul and score questions and does not generate
text.
This entry installs a converted redistribution: the CLM-v0.1-8B heads as
head.safetensors next to the unchanged Qwen3-8B backbone and tokenizer,
with config.json naming the ClmModel architecture. The checkpoint only
works with vllm.cpp (the vllm-cpp backend); transformers, vLLM and
llama.cpp cannot load it.
In LocalAI, serve it via POST /v1/systemone. Put the question in
instructions: the state head reads the state followed by the
instructions. The vllm.cpp project compared this checkpoint on CPU with the
reference code over eight questions: 8 of 8 answers agree, largest
probability difference 0.029 against the reference in bf16 (transformers
stood in for the reference's GPU encoder). This entry was installed and
served through LocalAI on CPU and gave the model card's example answer
(person, 0.950). There is no accuracy benchmark and no GPU run.
The entry sets a 4096-token context and a KV pool that holds four such
sequences. BF16 backbone with F32 heads, about 16.5 GB, pinned to a
revision. On CPU the model needs about 19 GB of free RAM (measured peak
17.9 GB resident).
Repository: localaiLicense: apache-2.0