Average Ratings 0 Ratings
Average Ratings 0 Ratings
Description
BaseRT offers a robust inference runtime for LLMs specifically optimized for Apple Silicon, allowing developers to seamlessly access models from Hugging Face, engage in local conversations, or utilize an API compatible with OpenAI through a single command-line interface. Enhanced by meticulously crafted Metal kernels, BaseRT aims to provide exceptional prefill and decoding efficiency on M-series Macs, with benchmark results indicating it performs up to 6.4 times faster in prefill tasks compared to llama.cpp, 3.9 times faster than MLX, and achieves a decoding speed that is 1.33 times quicker. The basert CLI is equipped to manage tasks such as model downloading, conversion, interactive chat, serving capabilities, completion generation, benchmarking, inspection, and bundle signing. Its server functionalities are extensive, encompassing chat interactions, text completions, embeddings, transcription services, tool calls, continuous batching, paged key-value caching, and prefix caching, with support for models that can handle text, vision, and audio data. BaseRT employs a proprietary .base model format that incorporates Q2–Q8 affine quantization, optional AWQ calibration, and signed bundles, and it is capable of converting GGUF, Hugging Face, and MLX checkpoints. Furthermore, this innovative runtime is tailored to maximize the capabilities of Apple Silicon, making it an essential tool for developers in the AI space.
Description
NVIDIA's Nemotron 3.5 Lightning is a state-of-the-art mixture-of-experts model boasting 30 billion parameters, of which 3 billion are actively utilized, specifically engineered for efficient, high-throughput performance in long-duration and continuously operating AI agents. This model is tailored for the execution components of agentic systems, adeptly managing frequent operations like tool invocations, output verification, routine commands, and delegating tasks to subagents, while larger reasoning models concentrate on strategic planning and orchestration. By employing a mixture-of-experts architecture, it activates only a select subset of parameters for each input token, marrying the expansive capacity of a larger model with significantly reduced computational demands. The training of this model is optimized for widely used agent harnesses and enhances inference speed through techniques such as speculative decoding, multi-token prediction, DFlash, and DSpark, making it versatile across various operational scenarios. Additionally, it is compatible with BF16 and NVFP4 checkpoints, providing flexibility in deployment from local systems like DGX Spark and GeForce RTX hardware to extensive data center infrastructures. In summary, its innovative design and scalability make it a powerful tool for advancing AI capabilities.
API Access
Has API
Yes
API Access
Has API
No
Integrations
Gemma 3
Yes
Gemma 4
Yes
Hermes Agent
No
Hugging Face
Yes
Llama 3.1
Yes
Llama 3.2
Yes
Mistral AI
Yes
NVIDIA NemoClaw
No
OpenAI
Yes
OpenClaw
No
Integrations
Gemma 3
No
Gemma 4
No
Hermes Agent
Yes
Hugging Face
No
Llama 3.1
No
Llama 3.2
No
Mistral AI
No
NVIDIA NemoClaw
Yes
OpenAI
No
OpenClaw
Yes
Pricing Details
No price information available.
Free Trial
No
Free Version
No
Pricing Details
No price information available.
Free Trial
No
Free Version
No
Deployment
Web-Based
No
On-Premises
No
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
Yes
Linux
No
Chromebook
No
Deployment
Web-Based
Yes
On-Premises
No
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
No
Linux
No
Chromebook
No
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
Yes
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
Yes
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Vendor Details
Company Name
Base Compute
Founded
2026
Country
Australia
Website
www.basecompute.co/getbasert
Vendor Details
Company Name
NVIDIA
Founded
1993
Country
United States
Website
nvidia.com