COG-GTM/llama.cpp

Support for Sparse MoE models like Camelidae and Sparsetral 7 days ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llama : add CLI assistant about 2 months ago

enhancement good first issue roadmap

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llama : integer type consistency in `llama.h` 4 months ago

enhancement good first issue refactoring roadmap

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llama cpp server cant open to public 6 months ago

enhancement good first issue server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: Mapping model name to LoRA config 7 months ago

enhancement good first issue server

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

kubernetes example 8 months ago

enhancement help wanted server/webui kubernetes

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server : add "token healing" support 8 months ago

enhancement good first issue roadmap server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Study how LM Evaluation Harness works and try to implement it 8 months ago

enhancement help wanted research 🔬 high priority generation quality

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Test replit-code-v1-3b model 8 months ago

help wanted model

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Support CoreML like whisper.cpp? 8 months ago

help wanted performance macos

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

feature request - disabling tokenizer in conversion / inference 8 months ago

enhancement good first issue help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Eos and bos tokens can be redefined as additional tokens with other ids 8 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

[IDEA] Global token enhancement/depression 8 months ago

help wanted research 🔬

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

mpi : attempt inference of 65B LLaMA on a cluster of Raspberry Pis 8 months ago

help wanted research 🔬 🦙. hardware

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Using MPI w/ 65b model but each node uses the full RAM. 8 months ago

help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

CUDA non-determinism on identical requests 8 months ago

bug good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llama : combined beam search + grammar sampling strategy 8 months ago

good first issue roadmap research 🔬 generation quality

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Request: Nougat OCR Integration 8 months ago

help wanted model

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

convert.py incorrectly detects LLaMAv1 65B as a LLaMAv2 model 8 months ago

good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ci : fix Docker workflow 8 months ago

help wanted build

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

GGUF endianness cannot be determined from GGUF itself 8 months ago

enhancement good first issue breaking change

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server : improvements and maintenance 8 months ago

help wanted refactoring roadmap server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

in situ auto-Frankenmerges 8 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Include `eos_token` and `bos_token` from `tokenizer_config.json` for chat templating 8 months ago

enhancement help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llama : refactor the llm.build_xxx functions 8 months ago

good first issue refactoring roadmap

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

MiniCPM 2b model support? 8 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llava-cli: improve llava-cli and the API for using LLaVA 8 months ago

enhancement good first issue help wanted llava

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

split: allow --split-max-size option 8 months ago

enhancement good first issue help wanted split

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

split: include the option in ./convert.py and quantize 8 months ago

enhancement good first issue help wanted split

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: exit failure if `--embedding` is set with an incoherent `--ubatch-size` 8 months ago

enhancement good first issue help wanted server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Docker: Embedding Issue & possible Fix 8 months ago

bug good first issue server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: doc: document the `--defrag-thold` option 8 months ago

documentation enhancement help wanted server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: support control vectors 8 months ago

enhancement good first issue server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Question: How to generate an MPS gputrace 8 months ago

help wanted high priority

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

common: download from URL, improve parallel download progress status 8 months ago

enhancement good first issue help wanted split

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Server: Add prompt processing progress endpoint? 8 months ago

enhancement help wanted server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: process prompt fairly accross slots 8 months ago

enhancement good first issue help wanted server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Inference not running when using a tokenizer with word model 8 months ago

bug help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ggml : add GPU support for Mamba models 8 months ago

enhancement help wanted roadmap Nvidia GPU

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

tts : add support for Orpheus 8 months ago

good first issue devin tts

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Support for Phi-3 models 8 months ago

good first issue model

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: avoid full prompt eval when 'prompt >= ctx' 8 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Llava functions compiled as extern "C" throw exceptions 8 months ago

bug good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Refactor: Existing examples refactoring opportunities 8 months ago

help wanted refactoring

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ggml : add DirectML backend 8 months ago

help wanted roadmap research 🔬

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ggml : add WebGPU backend 8 months ago

help wanted roadmap research 🔬

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: Installable package via winget 8 months ago

enhancement help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Improve `cvector-generator` 8 months ago

enhancement good first issue help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Refactor: Add more typechecking to GGUFWriter.add_key_value 8 months ago

help wanted refactoring

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Bug: Failed to run qwen2-57b-a14b-instruct-fp16. 8 months ago

bug good first issue high severity

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Bug: Unexpected output length (Only one token response!) when set configs "-n -2 -c 256" for llama-server 8 months ago

bug good first issue low severity

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ggml : add ANE backend 8 months ago

help wanted roadmap research 🔬

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: llama-server hot swapping cvectors via API like we can do with LoRA adapters now 8 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Misc. bug: server provides strutured output for response_format: json_object, but not for response_format: json_schema 8 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Compile bug: ios swift xcode build error when upgrade to llama : use cmake for swift build 8 months ago

good first issue help wanted build

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: Allow Filtering LLama Server Response Fields 8 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

examples : add configuration presets 8 months ago

documentation enhancement good first issue help wanted examples

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server : crash when -b > -ub with embeddings 8 months ago

bug good first issue server embeddings

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

tutorials : list for llama.cpp 8 months ago

good first issue help wanted roadmap

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: Implement missing ops from backends 8 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Open Issues Need Help