MPT-Chat (7B) API

Basic Information

• Version: Initial release, with subsequent variant models including MPT-7B-Chat, MPT-7B-Instruct, and MPT-7B-StoryWriter-65k+
• Model Type: Decoder-style Transformer, part of the GPT-style large language model family

Overview: MPT-7B represents MosaicML's leap into the open-source domain, aiming to democratize access to state-of-the-art transformer technology. It's designed for both generic and specific NLP tasks, with a particular emphasis on handling extremely long input sequences.

Key Features:

• Commercially Usable and Open Source: Licensed under Apache-2.0 for base and some variants, enabling wide accessibility and commercial application.
• Long Input Sequences: Utilizes ALiBi to manage unprecedented input lengths (up to 65k tokens), making it ideal for detailed text analysis and generation tasks.
• High Efficiency: Incorporates FlashAttention and FasterTransformer for accelerated training and inference, significantly reducing operational costs.
• Broad Accessibility: Integrated with HuggingFace for easy implementation, ensuring compatibility with existing machine learning workflows.

Intended Use:

The model is versatile, suitable for tasks ranging from machine learning research and application development to specific commercial uses in fields like tech and entertainment. Its variants are optimized for roles like conversational AI, narrative generation, and compliance with complex instructions.

Language Support:

Focused on English, incorporating a diverse array of text types, including technical and creative writing, to ensure robust language understanding.

Architecture:

Built as a decoder-only transformer with a configuration of 6.7 billion parameters, tailored for deep contextual understanding and generation.

Training Data:

The model's robustness stems from its training on 1 trillion tokens derived from a meticulously curated dataset combining text and code, ensuring a comprehensive linguistic and contextual grasp.

Use cases

• AI chatbots for multi-turn, context-aware dialogue and customer support
• interactive conversational assistants for productivity and content generation
• instruction-following for a wide variety of NLP tasks
• text summarization and long-form content generation (especially with extended context variants)
• coding assistance and code generation support
• academic, research and experimentation use in scalable open-source LLMs

Capabilities

• Function calling
• Streaming
• Structured outputs