Bonsai Demo

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

Generate images locally

OpenSSA: Small Specialist Agents based on Domain-Aware Neurosymbolic Agent (DANA) architecture for industrial problem-solving

Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Models

[CVPR 2024] Asymmetric Masked Distillation for Pre-Training Small Foundation Models

This repository features a custom-built decoder-only language model (LLM) with a total of 37 million parameters 🔥. I train the model to be able to ask question from a given context

Coding agent built around small (<10B) models for local development with a Rust(Actix Web, Diesel)/Typescript (Solid, Tailwind) opinionated stack. No coding knowledge needed. Multiple agents and roles - Product Owner, PM, Engineering Manager, Rust Engineer, SolidJS Engineer, etc. work together to build full-stack apps.

BobVLM – A 1.5B multimodal model built from scratch and pre-trained on a single P100 GPU capable of image descriptions and moderate question answering. 🤗🎉

Overview of self-supervised learning of tiny models, including distillation-based methods (aks. self-supervised distillation) and non-distillation methods.

Code for "On the Surprising Efficacy of Distillation as an Alternative to Pre-Training Small Models"

Provide specialized AI agents that develop, review, debug, and deploy production-ready code efficiently across various programming tasks.

Open-source harness distillation: frontier teachers improve prompts, tools, validators, skills, and runtime policies around weaker models.

If OpenAI builds engines, VOID builds the chassis.

Help us define the Pareto front of small models for MNIST classification. Frugal AI.

Tiny local AI agent for edge devices, built to make small models useful with workflows, personas, file/web context, and safe tool-like actions.

Manage Hermes CLI skins in a web UI, edit live YAML fields, and save or activate skins with built-in previews.

quickthink is a local-first CLI and Python library that wraps Ollama-backed LLM calls with a compressed plan-then-answer scaffold and latency-aware routing. It adds a short validated planning step for multi-step prompts and routes simple ones straight through. Local inference control for small models.

Hackathon starter kit for local LLMs (Gemma 4 by default): 6 tracks covering evals, agentic harnesses, RAG, quantization, multi-agent orchestration with ZeroClaw, and corpus-grounded Q&A.