The Avant-Garde of local Large Language Model
ek15072809 is an independent researcher and developer building avant-garde yet practical Large Language Models, aiming to realize AGI within a CPU / 16GB DRAM environment.
Areas of research and public technical work include Ta-Quantization, which achieves performance equivalent to roughly 4.5-bit quantization at 2.5-bit, on-demand SSD loading for MoE models, and lightweight agent tooling.
I am also aiming to submit work to arXiv's cs.CL / cs.LG categories. If you have a category you could recommend, I would be grateful to hear from you.
Research details are collected on the Research page, and update records are collected on the Commit page.
huggingface.co/TaQuants
TaQuants is an extreme low-bit quantization method that identifies and protects tensors that are vulnerable to quantization. In low-resource environments without a GPU, it achieves quality that surpasses importance-matrix-based approaches. Verification models and execution scripts are publicly available.
github.com/ek15072809/Swap-MoE
Swap-MoE is an implementation of on-demand SSD loading for MoE models on top of llama.cpp. By loading experts stored on SSD on an as-needed basis, it enables inference of 200B+ parameter models on a device with 16GB of DRAM.
github.com/ek15072809/Tema_Q-Agent
Tema_Q-Agent is a coding agent designed to run on local devices. Combined with Tqma_Q X6 TaIQ2_M, it makes it possible to build a highly capable agent tool that runs even in constrained compute environments.