Mistral 7B Instruct v0.3: new, advanced, instruction-based language model with enhanced features.
Model Name: Mistral-7B-Instruct-v0.3
Developer/Creator: Mistral AI in collaboration with Hugging Face
Release Date: 05/22/2024
Version: v0.3 latest
Model Type: Chat
The Mistral-7B-Instruct-v0.3 is an advanced version of the Mistral-7B model, fine-tuned specifically for instruction-based tasks. This model is designed to enhance language generation and understanding capabilities.
Extended Vocabulary: Supports up to 32,768 tokens for a diverse range of language inputs.
Version 3 Tokenizer: Improves language processing efficiency and accuracy.
Function Calling: Allows execution of predefined functions during language processing.
Instruction Fine-Tuning: Tailored for instruction-based tasks, improving contextual responses.
The model is designed for a variety of scenarios including:
Supports multiple languages due to its extended vocabulary and advanced tokenizer.
The Mistral-7B-Instruct-v0.3 is based on a transformer architecture. It employs grouped-query attention (GQA) for faster inference and sliding window attention (SWA) to handle long sequences efficiently. Key parameters from Mistral-7B-v0.1 include:
The model was trained on a diverse dataset sourced from various domains to ensure broad knowledge and robust performance. The training data encompasses a wide range of text inputs to enhance its understanding and response capabilities.
The exact volume of training data is not specified, but it includes extensive datasets from common benchmarks and publicly available data to ensure comprehensive language coverage.
The model's knowledge is up to date as of the release date, 05/22/2024.
Efforts have been made to include diverse datasets to minimize biases, but users should remain cautious of potential biases due to the nature of the data sources.