Model Overview Card for Stable Diffusion 3

Basic Information

• Model Name: Stable Diffusion 3
• Developer/Creator: Stability AI
• Release Date: February 22, 2024
• Version: 3.0
• Model Type: Text-to-Image Generation

Description

Overview

Stable Diffusion 3 is an advanced text-to-image generation model that utilizes a Multimodal Diffusion Transformer (MMDiT) architecture to produce high-quality images from textual descriptions.

Key Features

• Improved text understanding and spelling capabilities
• Enhanced performance in multi-subject prompts
• Superior image quality compared to previous versions
• Resource-efficient with models ranging from 800M to 8B parameters
• Unprecedented text quality in generated images

Intended Use

Stable Diffusion 3 is designed for various applications, including:

• Generating artworks and designs
• Educational and creative tools
• Research on generative models

Language Support

The model supports multiple languages for text input, leveraging its advanced text understanding capabilities.

Technical Details

Architecture

Stable Diffusion 3 employs a Multimodal Diffusion Transformer (MMDiT) architecture, which combines a diffusion transformer with flow matching techniques. The model uses separate sets of weights for image and language representations, enabling improved text understanding and image generation.

Training Data

While specific details about the training data are not provided, Stable Diffusion models are typically trained on large datasets of image-text pairs. The model likely uses a subset of the LAION-5B database, similar to previous versions.

Data Source and Size

The exact size of the training data is not specified, but it is expected to be substantial, given the model's performance and capabilities.

Knowledge Cutoff

The knowledge cutoff date for Stable Diffusion 3 is not explicitly stated, but it is likely to be recent, considering its release date of February 22, 2024.

Diversity and Bias

Stability AI emphasizes responsible AI practices and has implemented safeguards to prevent misuse. However, specific details about diversity and bias in the training data are not provided.

Performance Metrics

Stable Diffusion 3 demonstrates superior performance compared to state-of-the-art text-to-image generation systems such as DALL·E 3, Midjourney v6, and Ideogram v1. Human preference evaluations show advancements in typography and prompt adherence.

Comparison to Other Models

• Accuracy: Stable Diffusion 3 shows improvements in multi-subject prompts and image quality compared to previous versions.
• Speed: The 8B parameter model can generate a 1024x1024 image in 34 seconds using 50 sampling steps on an RTX 4090 GPU.
• Robustness: The model demonstrates enhanced capabilities in handling complex prompts and generating diverse imagery.

Usage

Ethical Guidelines

Stability AI emphasizes safe and responsible AI practices. They have implemented safeguards throughout the development process and continue to collaborate with researchers and experts to improve the model's safety and integrity.

Licensing

Stable Diffusion 3 is released under the Stability Community License. It's free for research, non-commercial, and commercial use for organizations or individuals with less than $1M annual revenue. For companies above this threshold, an Enterprise license is required