Basic Information

• Model Name: Llama 3.2 90B Vision Instruct Turbo
• Developer/Creator: Meta
• Release Date: September 25, 2024
• Version: 3.2
• Model Type: Multimodal (Text and Image)

Description

Overview

Llama 3.2 90B Vision Instruct Turbo is a large-scale multimodal AI model capable of processing both text and images. It represents Meta's first foray into multimodal AI, offering advanced visual reasoning capabilities alongside powerful language processing.

Key Features

• Multimodal processing of text and images
• 90 billion parameters
• Long context length support (up to 128k tokens)
• Optimized transformer architecture
• Supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF)
• High-resolution image processing (up to 1120x1120 pixels)

Intended Use

The model is designed for a wide range of applications, including:

• Document-level understanding
• Interpretation of charts and graphs
• Image captioning
• Visual question answering
• Data extraction and processing
• Image comparison
• Personal visual assistance

Language Support

The model supports multiple languages, making it suitable for multilingual tasks and applications.

Technical Details

Architecture

Llama 3.2 90B Vision Instruct Turbo utilizes an optimized transformer architecture. For image processing, it employs separately trained image reasoning adaptor weights that are integrated with the core LLM weights through cross-attention.

Training Data

• Data Source and Size: 6 billion (image, text) pairs
• Knowledge Cutoff: December 2023

Performance Metrics

The model demonstrates strong performance across various benchmarks:

• Matches OpenAI's GPT-4o on chart understanding (ChartQA)
• Outperforms Anthropic's Claude 3 Opus and Google's Gemini 1.5 Pro on interpreting scientific diagrams (AI2D)

Comparison to Other Models

Llama 3.2 90B Vision Instruct Turbo competes with leading models like Claude 3 Haiku and GPT-4o-mini in image recognition and visual understanding tasks.

Usage

Code Samples

Ethical Guidelines

The model includes a new Llama guard safety model to ensure responsible and ethical use.

Licensing

Llama 3.2 90B Vision Instruct Turbo is available under the Llama 3.2 Community License, which allows for fine-tuning and specific applications while maintaining certain restrictions.

‍