Skip to content
@EmbeddedLLM

EmbeddedLLM

We are committed to making production-grade AI inference as accessible and reliable as electricity, powered by vLLM.

Pinned Loading

  1. JamAIBase JamAIBase Public

    The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate…

    Python 1.1k 47

  2. vllm vllm Public

    Forked from vllm-project/vllm

    vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs

    Python 94 7

  3. embeddedllm embeddedllm Public

    EmbeddedLLM: API server for Embedded Device Deployment. Currently support CUDA/OpenVINO/IpexLLM/DirectML/CPU

    Python 54 5

Repositories

Showing 10 of 87 repositories
  • vllm Public Forked from vllm-project/vllm

    vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs

    EmbeddedLLM/vllm's past year of commit activity
    Python 94 Apache-2.0 22,951 1 10 Updated Sep 26, 2026
  • skypilot Public Forked from skypilot-org/skypilot

    SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.

    EmbeddedLLM/skypilot's past year of commit activity
    Python 2 Apache-2.0 1,273 0 2 Updated Sep 26, 2026
  • TokenVisor-recipes Public

    Inference recipes

    EmbeddedLLM/TokenVisor-recipes's past year of commit activity
    Python 0 0 0 0 Updated Sep 26, 2026
  • pi-cliffcompaction Public

    CliffCompaction for Pi: mechanical, truncate-only context compaction

    EmbeddedLLM/pi-cliffcompaction's past year of commit activity
    TypeScript 0 MIT 0 0 0 Updated Sep 26, 2026
  • litellm Public Forked from BerriAI/litellm

    Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

    EmbeddedLLM/litellm's past year of commit activity
    Python 0 11,977 0 0 Updated Sep 26, 2026
  • agentic-api Public Forked from vllm-project/agentic-api

    Stateful API logic for agentic applications using vLLM

    EmbeddedLLM/agentic-api's past year of commit activity
    Rust 1 Apache-2.0 82 0 0 Updated Sep 25, 2026
  • JamAIBase Public

    The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on AI applications.

    EmbeddedLLM/JamAIBase's past year of commit activity
    Python 1,102 Apache-2.0 47 1 1 Updated Sep 24, 2026
  • Infera Public Forked from AMD-AGI/Infera

    More token goodput from frontier models. A distributed, SLA-aware serving mesh — disaggregated prefill/decode, KV-aware routing, and cache offload, tuned to your production SLA.

    EmbeddedLLM/Infera's past year of commit activity
    Python 0 MIT 6 0 0 Updated Sep 23, 2026
  • fastsafetensors Public Forked from foundation-model-stack/fastsafetensors

    High-performance safetensors model loader

    EmbeddedLLM/fastsafetensors's past year of commit activity
    Python 0 Apache-2.0 37 0 0 Updated Sep 22, 2026
  • recipes Public Forked from vllm-project/recipes

    Common recipes to run vLLM

    EmbeddedLLM/recipes's past year of commit activity
    JavaScript 0 Apache-2.0 436 0 0 Updated Sep 21, 2026
View all repositories

Most used topics

llm llm-inference