o1-preview vs GPT-4o for complex reasoning - worth the cost?

Dr. Anna KowalskiDec 5, 2024

I've been testing o1-preview for research-level reasoning tasks and comparing it with GPT-4o. The cost difference is significant ($15/M input for o1 vs $2.50/M for GPT-4o), so I want to make sure it's justified.

My test cases include:

Multi-step mathematical proofs

Complex code debugging (finding subtle logic errors)

Scientific paper analysis

Legal contract review

Preliminary findings:

o1 dramatically outperforms on math proofs (92% vs 64% accuracy)

For code debugging, the gap is smaller (85% vs 71%)

For text analysis tasks, GPT-4o is often sufficient

o1's "thinking" tokens add up fast — some requests use 10K+ reasoning tokens

My current strategy: use GPT-4o as default, escalate to o1 only for tasks that require multi-step logical reasoning. Anyone else using a similar tiered approach?

7.8k views39 replies98 likes

2 Replies

Logan K.StaffAccepted AnswerDec 6

You need to pass the proxy directly to the httpx client:

import httpx
from openai import OpenAIclient = OpenAI(
    http_client=httpx.Client(
        proxy="http://proxy.corp.example.com:8080",
        verify=False  # Only if your proxy uses self-signed certs
    )
)

The environment variables aren't picked up by the SDK's default httpx client configuration.

Robert Chang Dec 7

This worked perfectly. For async usage:

client = AsyncOpenAI(
    http_client=httpx.AsyncClient(
        proxy="http://proxy.corp.example.com:8080"
    )
)