Average Ratings 0 Ratings
Average Ratings 0 Ratings
Description
BaseRT offers a robust inference runtime for LLMs specifically optimized for Apple Silicon, allowing developers to seamlessly access models from Hugging Face, engage in local conversations, or utilize an API compatible with OpenAI through a single command-line interface. Enhanced by meticulously crafted Metal kernels, BaseRT aims to provide exceptional prefill and decoding efficiency on M-series Macs, with benchmark results indicating it performs up to 6.4 times faster in prefill tasks compared to llama.cpp, 3.9 times faster than MLX, and achieves a decoding speed that is 1.33 times quicker. The basert CLI is equipped to manage tasks such as model downloading, conversion, interactive chat, serving capabilities, completion generation, benchmarking, inspection, and bundle signing. Its server functionalities are extensive, encompassing chat interactions, text completions, embeddings, transcription services, tool calls, continuous batching, paged key-value caching, and prefix caching, with support for models that can handle text, vision, and audio data. BaseRT employs a proprietary .base model format that incorporates Q2–Q8 affine quantization, optional AWQ calibration, and signed bundles, and it is capable of converting GGUF, Hugging Face, and MLX checkpoints. Furthermore, this innovative runtime is tailored to maximize the capabilities of Apple Silicon, making it an essential tool for developers in the AI space.
Description
The Mixtral 8x7B model is an advanced sparse mixture of experts (SMoE) system that boasts open weights and is released under the Apache 2.0 license. This model demonstrates superior performance compared to Llama 2 70B across various benchmarks while achieving inference speeds that are six times faster. Recognized as the leading open-weight model with a flexible licensing framework, Mixtral also excels in terms of cost-efficiency and performance. Notably, it competes with and often surpasses GPT-3.5 in numerous established benchmarks, highlighting its significance in the field. Its combination of accessibility, speed, and effectiveness makes it a compelling choice for developers seeking high-performing AI solutions.
API Access
Has API
Yes
API Access
Has API
Yes
Integrations
302.AI
No
C++
No
Clojure
No
Cody
No
Continue
No
DataChain
No
Expanse
No
F#
No
GaiaNet
No
Gemma 3
Yes
Integrations
302.AI
Yes
C++
Yes
Clojure
Yes
Cody
Yes
Continue
Yes
DataChain
Yes
Expanse
Yes
F#
Yes
GaiaNet
Yes
Gemma 3
No
Pricing Details
No price information available.
Free Trial
No
Free Version
No
Pricing Details
Free
Open source
Free Trial
No
Free Version
Yes
Deployment
Web-Based
No
On-Premises
No
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
Yes
Linux
No
Chromebook
No
Deployment
Web-Based
No
On-Premises
Yes
iPhone App
No
iPad App
No
Android App
No
Windows
Yes
Mac
Yes
Linux
Yes
Chromebook
No
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
Yes
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
No
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Vendor Details
Company Name
Base Compute
Founded
2026
Country
Australia
Website
www.basecompute.co/getbasert
Vendor Details
Company Name
Mistral AI
Founded
2023
Country
France
Website
mistral.ai/news/mixtral-of-experts/