OpenAI Integration (Python)

If you use the OpenAI Python SDK, you can use the Langfuse drop-in replacement to get full logging by changing only the import.

- import openai
+ from langfuse.openai import openai

Langfuse automatically tracks:

All prompts/completions with support for streaming, async and functions
Latencies
API Errors (example)
Model usage (tokens) and cost (USD) (learn more)

Get started

1. Setup

The integration is compatible with OpenAI SDK versions >=0.27.8. It supports async functions and streaming for OpenAI SDK versions >=1.0.0.

%pip install langfuse openai --upgrade

import os
 
# get keys for your project from https://cloud.langfuse.com
os.environ["LANGFUSE_PUBLIC_KEY"] = ""
os.environ["LANGFUSE_SECRET_KEY"] = ""
 
# your openai key
os.environ["OPENAI_API_KEY"] = ""
 
# Your host, defaults to https://cloud.langfuse.com
# For US data region, set to "https://us.cloud.langfuse.com"
# os.environ["LANGFUSE_HOST"] = "http://localhost:3000"

2. Replace import

- import openai
+ from langfuse.openai import openai

# Optional, checks the SDK connection with the server. Not recommended for production usage.
from langfuse.openai import auth_check
 
auth_check()

Optional attributes

Instead of setting the environment variables before importing the SDK, you can also use the following attributes after the import. This works for the async OpenAI client as well:

Attribute	Description	Default value
`openai.langfuse_host`	Host of the Langfuse API	`LANGFUSE_HOST` environment variable, defaults to `"https://cloud.langfuse.com"`. Set to `"https://us.cloud.langfuse.com"` for US data region.
`openai.langfuse_public_key`	Public key of the Langfuse API	`LANGFUSE_PUBLIC_KEY` environment variable
`openai.langfuse_secret_key`	Private key of the Langfuse API	`LANGFUSE_SECRET_KEY` environment variable
`openai.langfuse_debug`	Debug mode of Langfuse SDK	`False`

3. Use SDK as usual

No changes required.

Check out the notebook for end-to-end examples of the integration:

Example Notebook

Troubleshooting

Queuing and batching of events

The Langfuse SDKs queue and batches events in the background to reduce the number of network requests and improve overall performance. In a long-running application, this works without any additional configuration.

If you are running a short-lived application, you need to flush Langfuse to ensure that all events are flushed before the application exits.

openai.flush_langfuse()

Learn more about queuing and batching of events here.

Advanced usage

Custom trace properties

You can add the following properties to the openai method, e.g. openai.chat.completions.create(), to use additional Langfuse features:

Property	Description
`name`	Set `name` to identify a specific type of generation.
`metadata`	Set `metadata` with additional information that you want to see in Langfuse.
`session_id`	The current session.
`user_id`	The current user_id.
`tags`	Set tags to categorize and filter traces.
`trace_id`	See "Interoperability with Langfuse Python SDK" (below) for more details.
`parent_observation_id`	See "Interoperability with Langfuse Python SDK" (below) for more details.

Example:

openai.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
      {"role": "system", "content": "You are a very accurate calculator. You output only the result of the calculation."},
      {"role": "user", "content": "1 + 1 = "}],
    name="test-chat",
    metadata={"someMetadataKey": "someValue"},
)

Group multiple LLM calls into a single trace

Usually, this integration creates a single trace in Langfuse for each openai call. If you want to group multiple openai calls into a single trace, you can use the trace_id property. This can be either a random id or an existing id from your application that will be unique to this execution.

from langfuse.openai import openai
from uuid import uuid4
trace_id = str(uuid4())
 
country = "Bulgaria"
 
capital = openai.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
      {"role": "system", "content": "What is the capital of the country?"},
      {"role": "user", "content": country}],
  temperature=0,
  name="get-capital",
  trace_id=trace_id
).choices[0].message.content
 
poem = openai.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
      {"role": "system", "content": "You are a poet. Create a poem about this city."},
      {"role": "user", "content": capital}],
  name="generate-poem",
  trace_id=trace_id
).choices[0].message.content

Now both generations are grouped in the same trace with the trace_id assigned by you.

TRACE (id: trace_id)
|
|-- GENERATION: get-capital
|
|-- GENERATION: generate-poem

If you want to add further properties or observations to the trace, read the next section.

Interoperability with Langfuse Python SDK

Use the OpenAI integration in combination with the regular Langfuse SDKs (full reference) if you want to:

Add non-OpenAI related observations to the trace.
Group multiple OpenAI calls into a single trace while customizing the trace.
Have more control over the trace structure.

ℹ️

New to Langfuse Tracing? Checkout this introduction to the basic concepts.

Desired trace structure:

TRACE: capital-poem-generator
|
|-- GENERATION: get-capital
|
|-- GENERATION: generate-poem

Implementation:

from langfuse import Langfuse
from langfuse.openai import openai
 
# initialize SDK
langfuse = Langfuse()
 
# create trace and add params
trace = langfuse.trace(name="capital-poem-generator")
 
# create multiple completions, pass trace_id to each
 
country = "Bulgaria"
 
capital = openai.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
      {"role": "system", "content": "What is the capital of the country?"},
      {"role": "user", "content": country}],
  name="get-capital",
  trace_id=trace_id
).choices[0].message.content
 
poem = openai.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
      {"role": "system", "content": "You are a poet. Create a poem about this city."},
      {"role": "user", "content": capital}],
  name="generate-poem",
  trace_id=trace_id
).choices[0].message.content