​
Understanding Sessions

A session groups traces using a session ID attribute. This is particularly useful when developing or debugging a chatbot application, as it allows you to view collections of messages or traces that belong to a series of interactions between a human and the AI. By incorporating session.id and user.id as span attributes, you can:

  • Pinpoint where a conversation “breaks” or deviates. This helps in identifying if a user becomes increasingly frustrated or if a chatbot is ineffective.
  • Identify trace groups where your application underperforms. Adding session.id and/or user.id from an application allows for grouping and further filtering of interactions.
  • Develop custom metrics based on evaluations using session.id or user.id to identify the best and worst performing sessions and users.

​
Adding SessionID and UserID

Session and user IDs can be added to a span through auto instrumentation or manual instrumentation of traceAI. Any LLM call within the context (the with block in the example below) will include the corresponding session.id or user.id as a span attribute. Both session.id and user.id must be non-empty strings.

When setting up your instrumentation, you can pass the sessionID attribute as demonstrated below.

​
using_session

This context manager adds a session ID to the current OpenTelemetry Context. traceAI auto instrumentators will read this Context and pass the session ID as a span attribute, adhering to the traceAI semantic conventions. The session ID input must be a non-empty string.

from fi_instrumentation import using_session

with using_session(session_id="my-session-id"):
    # Calls within this block will generate spans with the attributes:
    # "session.id" = "my-session-id"
    ...

It can also be applied as a decorator:

Python
@using_session(session_id="my-session-id")
def call_fn(*args, **kwargs):
    # Calls within this function will generate spans with the attributes:
    # "session.id" = "my-session-id"
    ...

​
using_user

This context manager adds a user ID to the current OpenTelemetry Context. traceAI auto instrumentators will read this Context and pass the user ID as a span attribute, following the traceAI semantic conventions. The user ID input must be a non-empty string.

from fi_instrumentation import using_user

with using_user("my-user-id"):
    # Calls within this block will generate spans with the attributes:
    # "user.id" = "my-user-id"
    ...

It can also be applied as a decorator:

Python
@using_user("my-user-id")
def call_fn(*args, **kwargs):
    # Calls within this function will generate spans with the attributes:
    # "user.id" = "my-user-id"
    ...

​
Additional Examples

Install the required package:

pip install traceAI-openai

Once your OpenAI client is defined, any call inside our context managers will attach the corresponding attributes to the spans.

import openai
from fi_instrumentation import using_attributes

client = openai.OpenAI()

# Defining a Session
with using_attributes(session_id="my-session-id"):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Write a haiku."}],
        max_tokens=20,
    )

​
Defining a User

# Ensure 'client' and 'using_attributes' are imported as in the previous Python example.
with using_attributes(user_id="my-user-id"):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Write a haiku."}],
        max_tokens=20,
    )

​
Defining a Session AND a User

# Ensure 'client' and 'using_attributes' are imported as in the previous Python example.
with using_attributes(
    session_id="my-session-id",
    user_id="my-user-id",
):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Write a haiku."}],
        max_tokens=20,
    )

Alternatively, if you wrap your calls inside functions, you can use them as decorators:

Python
from fi_instrumentation import using_attributes

client = openai.OpenAI()

# Defining a Session
@using_attributes(session_id="my-session-id")
def call_fn(client, *args, **kwargs):
    return client.chat.completions.create(*args, **kwargs)
    
# Defining a User
@using_attributes(user_id="my-user-id")
def call_fn(client, *args, **kwargs):
    return client.chat.completions.create(*args, **kwargs)

# Defining a Session AND a User
@using_attributes(
    session_id="my-session-id",
    user_id="my-user-id",
)
def call_fn(client, *args, **kwargs):
    return client.chat.completions.create(*args, **kwargs)