Model Overview Card for GPT-4o Mini Audio‍

Basic Information

• Model Name: GPT-4o Mini Audio
• Developer/Creator: OpenAI
• Model Type: Voice Generation
• Price: Text input $0.15; output $0.63
             Audio input $10.50; output $21

Description

Overview

Designed for quick, low-resource speech applications, GPT-4o Mini Audio enables fast, natural interactions in audio-based tools with support for both speech input and output. It is a cost-effective version that offers advanced audio capabilities at just 25% of the cost of the full GPT-4o Audio models, making it accessible for developers building voice-driven applications.

Key Features

• Real-Time Voice Interaction: Processes and generates voice and text responses
• Lightweight Deployment: Fits in resource-constrained environments
• Multilingual Audio Support: Speech recognition in 50+ languages
• Fast Response Time: Low latency interactions
• Cost Efficiency: Operates at 25% of the cost of GPT-4o Audio models, ideal for budget-conscious applications

Intended Use

• Voice Assistants on Mobile: Low-resource smart agents
• Accessibility Features: Voice control and feedback
• Embedded IoT Tools: Smart devices with audio AI

Technical Details

Architecture

Derived from GPT-4o through model distillation, it retains the Transformer-based architecture optimized for audio tasks. The model includes advanced voice activity detection (VAD) layers for precise audio segmentation and processing.

Training Data

The model was trained on a diverse dataset that includes:

• Multilingual speech corpora.
• Synthetic voice data for various accents and tones.
• Publicly available audiobooks, podcasts, and conversational datasets.

Data Source and Size

The training data spans hundreds of hours of high-quality audio recordings combined with billions of text tokens to ensure robust multimodal performance.

Knowledge Cutoff

October 2023, with no real-time web search capability but optimized for static datasets.

Performance Metrics

Accuracy

Achieves high-rate performance in:

• Speech-to-text transcription with a Word Error Rate (WER) of 6.5%.
• Text-to-audio synthesis with high fidelity and natural intonation scores above 92%.

Speed

Processes asynchronous audio tasks at an average latency of 420 milliseconds per second of input audio, making it suitable for near-real-time applications.

Robustness

Handles diverse accents, dialects, and noisy environments effectively but may exhibit reduced accuracy in highly specialized jargon or low-resource languages.

Usage

Code Samples

The model is available on the AI/ML API platform as "gpt-4o-mini-audio".

API Documentation

Detailed API Documentation is available on the AI/ML API website, providing comprehensive guidelines for integration

Ethical Guidelines

OpenAI has established ethical considerations in the model's development, focusing on safety and bias mitigation. The model incorporates OpenAI’s bias mitigation framework but may reflect biases inherent in its training data sources, particularly in underrepresented languages or accents.

Licensing

GPT-4o is available under commercial usage rights, allowing businesses to integrate the model into their applications.