Home › Model Serving › New API

New API - AI Model Serving Tool

Overview

New API is an open-source, next-generation LLM gateway and AI asset management system that standardizes access to multiple large model APIs. It provides a rich UI, multi-language support, online recharge, usage tracking, token grouping, model charging, and configurable reasoning effort for personal and enterprise internal management and distribution.

Key Features

  • Standardized interface for multiple large model APIs such as OpenAI and Claude
  • Rich web-based user interface
  • Multi-language support
  • Online recharge and billing controls
  • Usage tracking and token grouping
  • Model-level charging configuration
  • Configurable reasoning effort per model or request
  • Designed for personal and enterprise internal distribution

Ideal Use Cases

  • Unify calls to multiple LLM providers for applications
  • Centralize internal AI assets and model access control
  • Track and allocate usage and costs across teams
  • Offer paid access or charging per model or request
  • Provide a configurable gateway for enterprise deployments

Getting Started

  • Open the GitHub repository: https://github.com/QuantumNous/new-api
  • Clone or download the repository locally
  • Review the project's README and configuration documentation
  • Configure external model provider API keys and mappings
  • Deploy or run the provided web UI
  • Enable online recharge, billing, and usage tracking features
  • Test endpoints and verify token and usage reporting

Pricing

Not disclosed in source; repository is available on GitHub.

Key Information

  • Category: Model Serving
  • Type: AI Model Serving Tool
Visit Official Website

Related Tools

Replicate

A platform that allows users to run, host, and share AI models via an API, supporting a variety of generative tasks.
Hugging Face

A robust AI platform where the machine learning community collaborates on models, datasets, and applications.
Hugging Face Spaces

A platform for hosting machine learning demo apps with support for GPU acceleration, Docker, and custom Python environments.
HUGS

Optimized, zero‐configuration inference microservices from Hugging Face designed to simplify and accelerate the deployment of open AI models via an OpenAI‐compatible API.
OpenVINO

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference across various platforms. It supports models trained with popular frameworks and enhances performance for deep learning tasks in computer vision, automatic speech recognition, and natural language processing.
Hugging Face Hub

The official Python client for the Hugging Face Hub, allowing users to interact with pre-trained models and datasets, manage repositories, and run inference on deployed models.