Retrieval Augmented Generation (RAG)

Logs a rag data with context for evaluation using Future Agi client.

Creating an Client

from fi_client import Client

api_key = os.environ["FI_API_KEY"]
secret_key = os.environ["FI_SECRET_KEY"]
base_url = os.environ["FI_API_URL"]

fi_client = Client(api_key=api_key, secret_key=secret_key,uri=base_url)

Supported Model Types:

• GENERATIVE_LLM: For text data.

Supported Environments:

• TRAINING: For models in the training phase.

• VALIDATION: For models in the validation phase.

• PRODUCTION: For models deployed in a production environment.

• CORPUS: For models dealing with a large collection of data or corpus.

Sending a data for Evaluation

To log an event, you must provide required parameters like model_id, model_type, environment, and conversation, with optional fields like model_version, prediction_timestamp, and tags. It’s important to log each data entry sequentially, one by one.

from fi.client import ModelTypes, Environments
import time

fi_client.log(
  model_id = "your-model-ID",
  model_type = ModelTypes.GENERATIVE_LLM,
  environment = Environments.PRODUCTION,
  model_version = "v2",
  prediction_timestamp =int(time.time()),
  consersation = {
    "chat_graph": {
        "conversation_id": "123456",
        "nodes": [
            
            {
                "node_id": "node-1" ,
                "parent_id": None ,
                "timestamp": "" ,
                "message": {
                    "id": "message-1",
                    "author": {
                        "role": "user",
                        "metadata": {}
                    },
                    "content": {
                        "content_type": "text",
                        "parts": [ <instructions> ]
                    },
                    "context": [ [str ,str] ] #format [ ["",""] , ["",""]]
                }
            },
            {   "node_id": "Node-2" ,
                "parent_id": "Node-1" ,
                "timestamp": "" ,
                "message": {
                    "id": "message-2",
                    "author": {
                        "role": "assistant",
                        "metadata": {}
                    },
                    "content": {
                        "content_type": "text",
                        "parts": [ <instructions> ]
                    },
                    "context": [ [str ,str] ] #format [ ["",""] , ["",""]]
                }
            }
        ]
    }
}
).result()

Structure:

• conversation_id: The ID of the conversation. Must be a string.

• nodes: A list of nodes, each representing a message in the conversation graph.

• node_id: The ID of the node. Must be a string.

• parent_id: The ID of the parent node. Must be a string (or null for root nodes).

• timestamp: The timestamp of the node. Must be an integer.

• message: A dictionary with the message details.

• id: The ID of the message. Must be a string.

• author: Information about the message author.

• role: The role of the author (e.g., “user”, “assistant”). Must be a string.

• metadata: Additional metadata about the author.

• content: The content of the message.

• content_type: The type of content (e.g., “text”).

• parts: A list of content parts (e.g., text segments).

• context: (Optional) The context of the message. Must be a list of pairs of strings in the format [["", ""]...].

Description:

• Conversation ID: Unique identifier for the conversation.

• Nodes: List of messages in the conversation, structured as nodes in a graph.

• Node ID: Unique identifier for each node.

• Parent ID: Indicates the parent node, establishing the hierarchy. Root nodes have null as the parent ID.

• Timestamp: Time when the message was created.

• Message: Contains details of the message, including the author, content, and optional context.