o1-preview
VS
o1-mini

Comparing two of OpenAI's newest model,
pitting o1-preview and o1-mini in a head-to-head battle.
Let's explore, which model performs better for your usecase.

Get API Key

Benchmarks and specs

Specs

OpenAI recently introduced two new models: o1-preview and o1-mini. While Preview is designed to deliver superior reasoning performance, it comes with a higher cost. O1-mini in the meantime, is marketed as an ultimate STEM tool. In this article, we’ll explore when it makes sense to opt for the more powerful Preview model and when the more affordable Mini model will be sufficient for your needs. Let's consider the specs first.

Specification o1-preview o1-mini
Сontext Window 128K 128K
Output Tokens 65K 32K
Number of parameters in the LLM unknown unknown
Knowledge cutoff October 2023 October 2023
Release Date September 12, 2024 September 12, 2024
Tokens per second ~74 ~23

The main differences between o1-mini and o1-preview are their maximum output tokens and processing speeds. O1-mini can generate up to 65K tokens and operates at a speed of 73.9 tokens per second. In comparison, o1-preview is limited to 32K output tokens and processes at 23 tokens per second. The practical distinction between them depends on their performance in specific use cases.

Benchmarks

This benchmark is the combination of official release notes for both models, as well as multiple open benchmarks

Benchmark o1-preview o1-mini
Undergraduate level knowledge
MMLU
90.8 85.2
Graduate level reasoning
GPQA
73.3 60
Code
Human Eval
92.4 92.4
Codeforces competition
Codeforces
1258 1650
Cybersecurity
CTFs
43.0 28.7
Math problem-solving
MATH
85.5 90.0

Interestingly, o1-mini performs lower in Cybersec whilst being superior in coding. It still performs better in Maths. Let's dive into the practical test to see how the models perform under pressure.

Get API Key

Practical tests

We are rigorously testing AI in difficult, real-world scenarios. We will use prompts related to conversation, coding, and image analysis to assess the performance of each model.

Some of the outputs will be edited down for shortness or clarity.

Prompt 1. Just Chatting:

How many hearts in this series? ❤️👣🦷🫦💙♥️🧠🫁💔💗💛🧑🏿🦿

o1-preview output

o1-mini output

Summing up the hearts:
- ❤️: 1 heart
- 💙: 1 heart
- ♥️: 1 heart
- 💔: 2 hearts 
      (two separate halves)
- 💗: 3 hearts
      (main heart with two 
      smaller hearts around it)
- 💛: 1 heart

#Answer
There are 9 hearts depicted 
in the series
Let's count the heart-related 
emojis in your series:

1. ❤️ – Red Heart
2. 💙 – Blue Heart
3. ♥️ – Heart Suit
4. 💔 – Broken Heart
5. 💗 – Growing Heart
6. 💛 – Yellow Heart

#Answer
There are 6 heart emojis
in the series you provided.
Correct Answer  
4/5 times the models responded with a standard answer, giving the interesting response from above once. Very detailed analysis, albeit maybe a little too unconvential.
Correct Answer ✅

Prompt 2. Logical prompt:

What do these words have in common?
Freight Stone Often Canine.
No...not that they all contain an 'e'

o1-preview output

o1-mini output

The words "Freight," "Stone," 
"Often," and "Canine" 
all contain embedded number words:

- Freight contains eight (`fr**eight**`)...
- Stone contains one (`st**one**`)...
- Often contains ten (`of**ten**`)...
- Canine contains nine (`ca**nine**`)...

"Answer:"
Each word contains a number word
“one,” “ten,” “nine,” or “eight”
within it.
Each of the words contains the name
of a number embedded within it:

1. Freight – Contains "eight"
2. Stone – Contains "one"
3. Often – Contains "ten"
4. Canine – Contains "nine"

"Answer:"
So, the commonality is that 
each word includes a number 
(one, eight, nine, ten) hidden 
within its letters.
Correct Answer: ✅ 

Correct Answer: ✅ 
Surprisingly, both models have completed this test - the first ones to do so in our testing history.

Prompt 3. Logical reasoning:

Which of the following equations is the summary of photosynthesis? 

a. 6CO2 + 12H2O → C6H12O6 + 6O2 + 6H2O 
b. 6CO2 + H2O + Sunlight → C6H12O6 + O2 + 6H2O 
c. 6CO2 + 12H2O + Chlorophyll + Sunlight → C6H12O6 + 6O2 + 6H2O 
d. 6CO2 + 12H2O + Chlorophyll → C6H12O6 + 6CO2 + 6H2O

o1-preview output

o1-mini output

The correct answer is 'c'. 6CO2 + 12H2O + Chlorophyll + Sunlight -> C6H12O6 + 6O2 + 6H2O.

This equation accurately summarizes the process of photosynthesis, incorporating all necessary reactants and products, as well as the essential factors of chlorophyll and sunlight that drive the process.

Explanation:

Photosynthesis is the process by which green plants convert light energy into chemical energy, producing glucose (C6H12O6) and releasing oxygen (O2) as a by-product. The overall balanced chemical equation for photosynthesis can be represented as:

6CO2 + 12H2O + Chlorophyll + Sunlight -> C6H12O6 + 6O2 + 6H2O

1. Carbon Dioxide (6CO2): CO2 from the atmosphere is a raw material that provides carbon atoms for glucose synthesis.
2. Water (12H2O): Water absorbed by the roots is split during the light-dependent reactions. Some of it is used in forming glucose, and some is released back as water.
3. Chlorophyll: This pigment located in chloroplasts captures sunlight energy necessary for driving the reactions.
4. Sunlight: Provides the energy required to split water molecules and drive the synthesis of glucose.
5. Glucose (C6H12O6): The main product, a carbohydrate that stores energy for the plant.
6. Oxygen (6O2): Released as a by-product from the splitting of water molecules.
7. Water (6H2O): Some water is regenerated and released as a product.

Options a and b either omit essential components like chlorophyll and sunlight or misrepresent the stoichiometry of water and oxygen in the reaction.

Option d incorrectly shows CO2 as a product, which contradicts the essence of photosynthesis where CO2 is a reactant consumed by the plant.

'Answer:' 
c
The correct answer is:

a. 6CO₂ + 12H₂O → C₆H₁₂O₆ + 6O₂ + 6H₂O

Explanation:
Photosynthesis is the process by which green plants, algae, and some bacteria convert carbon dioxide and water into glucose and oxygen using sunlight as the energy source. The general balanced chemical equation for photosynthesis is:   

[ 6CO₂ + 6H₂O + \text{light energy} → C₆H₁₂O₆ + 6O₂ ]

However, when considering the overall process within the plant cells, water is both consumed and produced. Specifically, water is used in the light-dependent reactions, and some of it is regenerated in the Calvin cycle. This leads to a more detailed balanced equation:

[ 6CO₂ + 12H₂O → C₆H₁₂O₆ + 6O₂ + 6H₂O ]

- 6CO₂: Carbon dioxide taken in from the atmosphere.
- 12H₂O: Water absorbed by the plant from the soil.
- C₆H₁₂O₆: Glucose produced, which serves as an energy source for the plant.
- 6O₂: Oxygen released into the atmosphere as a byproduct.
- 6H₂O: Water regenerated during the process.

Why the Other Options Are Incorrect:
b. 6CO₂ + H₂O + Sunlight → C₆H₁₂O₆ + O₂ + 6H₂O
  - Incorrect Balance: The number of water molecules does not balance correctly (only 1 H₂O is initially consumed).

c. 6CO₂ + 12H₂O + Chlorophyll + Sunlight → C₆H₁₂O₆ + 6O₂ + 6H₂O
  - Extra Components: While chlorophyll and sunlight are essential for photosynthesis, the summary equation typically focuses on the reactants and products without specifying catalysts or energy sources.

d. 6CO₂ + 12H₂O + Chlorophyll → C₆H₁₂O₆ + 6CO₂ + 6H₂O
  - Incorrect Products: This equation incorrectly shows the production of additional CO₂, which is not a product of photosynthesis.

#Answer:
Option A best represents the summary of the photosynthesis process.
Correct Answer: ✅
Correct Answer: ❌

Prompt 4. Math:

Let A, B, C, and D be points on the hyperbola x^2/20 - y^2/24 = 1 
such that ABCD is a rhombus whose diagonals intersect at the origin. 
Find the greatest real number that is less than BD^2 for all such rhombi.

o1-preview output

o1-mini output

To solve this problem, we need to find the maximum value of BD^2 for the rhombus ABCD inscribed in the hyperbola x^2/20 - y^2/24 = 1 with diagonals intersecting at the origin.

We start by noting the properties of a rhombus:

1. All sides are equal in length.
2. The diagonals are perpendicular bisectors of each other.
3. The diagonals intersect at the origin.

Because the diagonals intersect at the origin and are perpendicular, they must lie along lines with slopes that are negative reciprocals of each other. Let’s define one diagonal along the line y = mx and the other diagonal along y = -1/mx, where m is the slope of the first diagonal.

The points where these lines intersect the hyperbola will give us the vertices of the rhombus. Substituting y = mx into the hyperbola’s equation provides us with the x-coordinates of points A and C:

x^2/20 - (mx)^2/24 = 1.

This simplifies to:

x^2(1/20 - m^2/24) = 1.

Similarly, substituting y = -1/mx gives us the x-coordinates of points B and D:

x^2(1/20 - 1/(m^2 * 24)) = 1.

From these equations, we can solve for the coordinates of B and D in terms of m.

Next, the length of diagonal BD is twice the distance from the origin to point B, since the rhombus is symmetric about the origin. Using the distance formula and the coordinates of B, we find:

BD^2 = 4(xB^2 + yB^2) = 4 xB^2 (1 + 1/m^2).

Substituting for xB^2 from the earlier equation, we get:

BD^2 = 4 * 1/β (1 + 1/m^2),

where β = 1/20 - 1/(m^2 * 24).

Simplifying, we obtain:

BD^2 = 4 a^2 b^2 (1 + m^2)/(m^2 b^2 - a^2).

To find the maximum value of BD^2, we analyze this function. By taking the derivative with respect to m^2 and setting it to zero, we find that this expression reaches a maximum when m^2 = 1.

Plugging m^2 = 1 back into the equation, along with a^2 = 20 and b^2 = 24, we find:

BD^2 = 8 a^2 b^2/(b^2 - a^2) = 8 * 20 * 24/(24 - 20) = 8 * 20 * 24/4 = 960.

'Answer:'
Therefore, the maximum value 
of BD^2 is 960.
To solve the problem, let's analyze the geometric conditions given:

1. Hyperbola Equation:
x^2/20 - y^2/24 = 1
Points A, B, C, and D lie on this hyperbola.

2. Rhombus Properties:
Opposite sides are equal.
Diagonals bisect each other at the origin (the midpoint).

3. Parametrizing the Points:
Let's denote:
A = (x1, y1), C = (-x1, -y1)
B = (x2, y2), D = (-x2, -y2)
Since ABCD is a rhombus, the diagonals AC and BD intersect at the origin, and:
AB = BC (distance equality)
This leads to the condition:
x1x2 + y1y2 = 0

4. Minimizing BD^2:
The length of diagonal BD is:
BD = 2sqrt(x2^2 + y2^2), BD^2 = 4(x2^2 + y2^2)
Subject to the hyperbola constraints:
x2^2/20 - y2^2/24 = 1
By solving the system of equations and applying the conditions, we find that:
BD^2 >= 480
The equality BD^2 = 480 cannot be achieved but can be approached as closely as desired.

5. Conclusion:
The greatest real number that is less than BD^2 for all such rhombi is 480.

#Answer:
480
Correct Answer: ❌ 
The model thought for a very long respons time and in the end it turned out to be incorrect.
Correct Answer: ✅ 
The answer was given relatively quickly (23 seconds), and most importantly, correctly.

Prompt 5. Let's get coding:

Create a game titled "Pixel Ninja Dash" using Python and Pygame, 
where all characters and obstacles are represented by rect objects. 
The game should follow the concept below and include 
the following features:
Game Concept:
Game Features:
Player Controls:
Endless Side-Scroller:
Obstacles:
Power-Ups:
Score System:
Difficulty Progression:
Lives:

With such tasks it is paramount to have very detailed requirements, which you can write out with an AI model itself.

o1-preview output

o1-mini output

snake game with a score counter, programmed by gpt 4osnake game, programmed by chat sonnet 3.5 with a counter, pause and a difficulty menu
Correct Answer: ❌
The game didn't even start.
Correct Answer: ✅ 
The game functions, but after jumping, the character falls below the screen and becomes unresponsive

Prompt 4:

Analyze the following image:
grand canyon with a river

LLama 3 70B output

ChatGPT-3.5 output

Clever trick!
You still have 4 marbles, 
but they're no longer in the cup 
because you turned it upside down! 

They're probably scattered 
around on the floor or counter now!
You still have 4 marbles in the cup, 
even though it is now upside down and
in the freezer
Correct Answer: ✅ 
Trick question deserves a trick answer! 
Good understading of nuance.
Correct Answer: ❌
Even Zero-shot Chain of Thought couldn't save it in tests.

Conclusion

Based on the benchmarks and practical tests, o1-mini and o1-preview exhibit distinct strengths and weaknesses. O1-mini excels in tasks requiring fast processing, precise math problem-solving, and competitive coding performance.

Get API Key

Pricing

O1-mini has pricing similar to previous frontier models, with o1-preview having having 5x the output price.

1M Tokens o1-preview o1-mini
Input price $15,75 $3,15
Output price $63 $12,6
Get API Key

Compare for yourself

You have already seen these models in action. Now it is your turn to evaluate them for your specific needs. Copy the code below into Google Colab or your preferred coding environment, add your API key, and begin experimenting!

import openai
import requests

def main():
    client = OpenAI(
      api_key=aiml_api_key,
      base_url="https://api.aimlapi.com",
    )

    # Specify the two models you want to compare
    model1 = 'o1-preview'
    model2 = 'o1-mini' 
    selected_models = [model1, model2]

    user_prompt = 'Why is the sky blue?'
    results= {}
    
    for model in selected_models:
        try:
            response = client.chat.completions.create(
                model=model,
                messages=[
                    {'role': 'user', 'content': "who is strong?"}
                ],
                max_tokens=2000,
            )
            print(response)
            message = response.choices[0].message.content
            results[model] = message
        except Exception as error:
            print(f"Error with model {model}:", error)

    # Compare the results
    print('Comparison of models:\n')
    print(f"{model1}:\n{results.get(model1, 'No response')}")
    print('\n')
    print(f"{model2}:\n{results.get(model2, 'No response')}")

if __name__ == "__main__":
    main()

Conclusion

The choice between GPT-1o Mini and GPT-1o Preview should be guided by specific needs:

  • GPT-1o Mini is ideal for tasks that demand fast processing, advanced mathematical solutions, and competitive coding scenarios.
  • GPT-1o Preview is better suited for in-depth reasoning, complex knowledge tasks, and creative coding prompts.

Each model's strengths and cost differences should also be considered in the decision-making process to ensure the most effective and cost-efficient use of the technology.

You can access both o1-mini and the models of o1-preview API, or see our full model lineup here - try for yourself, and get a feel for the frontier AI power!

Get API Key
many bubbles with ai logos, connected as a web

Access both models using our AI API

Explore and use multiple AI functionalities through a single API. Ideal for experimentation and groundbreaking projects.

200+ Models

3x faster response

OpenAI compatible

99.9% Uptime

AI Playground
Testimonials

Our Clients' Voices