Average Ratings 0 Ratings
Average Ratings 0 Ratings
Description
This repository showcases an implementation of model parallel autoregressive transformers utilizing GPUs, leveraging the capabilities of the DeepSpeed library. It serves as a record of EleutherAI's framework designed for training extensive language models on GPU architecture. Currently, it builds upon NVIDIA's Megatron Language Model, enhanced with advanced techniques from DeepSpeed alongside innovative optimizations. Our goal is to create a centralized hub for aggregating methodologies related to the training of large-scale autoregressive language models, thereby fostering accelerated research and development in the field of large-scale training. We believe that by providing these resources, we can significantly contribute to the progress of language model research.
Description
Ling 2.6 represents an independently developed and open-source series of large language models created by Ant Group, utilizing a Mixture of Experts (MoE) architecture to enhance inference efficiency, long context modeling, training methodologies, and collaborative reasoning for AI agents. By employing this MoE architecture, Ling effectively directs each token to engage only the most pertinent expert subnetworks, significantly reducing the computational load while preserving the extensive capabilities of the model. This series makes strides in long-sequence modeling, exemplified by Ling-2.6-1T, which accommodates a native context window of up to 1 million tokens and offers a 256K context window through its official API; additionally, Ling-2.6-flash features a native 256K context window, enabling it to handle around 200,000 characters in lengthy inputs. These models are meticulously crafted to ensure dependable retrieval of long-range information without any discernible loss of quality, regardless of whether the data is located at the start, middle, or end of the context. This innovative approach to long-context processing sets a new benchmark for efficiency and reliability in language model performance.
API Access
Has API
No
API Access
Has API
Yes
Integrations
Claude Code
No
Forefront
Yes
Hermes Agent
No
Kilo Code
No
OpenAI
No
OpenClaw
No
OpenRouter
No
ZBrain
Yes
Integrations
Claude Code
Yes
Forefront
No
Hermes Agent
Yes
Kilo Code
Yes
OpenAI
Yes
OpenClaw
Yes
OpenRouter
Yes
ZBrain
No
Pricing Details
Free
Open source
Free Trial
No
Free Version
Yes
Pricing Details
$0.0028 per 1M tokens
Free Trial
No
Free Version
No
Deployment
Web-Based
Yes
On-Premises
Yes
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
No
Linux
No
Chromebook
No
Deployment
Web-Based
Yes
On-Premises
No
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
No
Linux
No
Chromebook
No
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
No
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
Yes
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Vendor Details
Company Name
EleutherAI
Founded
2020
Website
github.com/EleutherAI/gpt-neox
Vendor Details
Company Name
Ant Group
Founded
2014
Country
China
Website
developer.ant-ling.com/en/docs/models/ling/