Package com.newrelic.api.agent

Interface AiMonitoring

  • public interface AiMonitoring
    This interface defines methods for recording LlmFeedbackMessage events and setting a callback for token calculation.

    • Method Detail

      • recordLlmFeedbackEvent

        void recordLlmFeedbackEvent​(java.util.Map<java.lang.String,​java.lang.Object> llmFeedbackEventAttributes)
        Records an LlmFeedbackMessage event.
        Parameters:
        llmFeedbackEventAttributes - A map containing the attributes of an LlmFeedbackMessage event. To construct the llmFeedbackEventAttributes map, use LlmFeedbackEventAttributes.Builder

        The map must include:

        • "traceId" (String): Trace ID where the chat completion related to the feedback event occurred
        • "rating" (Integer/String): Rating provided by an end user

        Optional attributes:

        • "category" (String): Category of the feedback as provided by the end user
        • "message" (String): Freeform text feedback from an end user
        • "metadata" (Map<String, String>): Set of key-value pairs to store additional data to submit with the feedback event

      • setLlmTokenCountCallback

        void setLlmTokenCountCallback​(LlmTokenCountCallback llmTokenCountCallback)
        Sets the callback function for calculating LLM tokens.
        Parameters:
        llmTokenCountCallback - The callback function to be invoked for counting LLM tokens. Example usage: 
        
                              LlmTokenCountCallback llmTokenCountCallback = new LlmTokenCountCallback() {
                                  {@literal @}Override
                                  public Integer calculateLlmTokenCount(String model, String content) {
                                      // Token calculation based on model and content goes here
                                      // Return the calculated token count
                                  }
                               };

                               // Set the created callback instance
                               NewRelic.getAgent().getAiMonitoring().setLlmTokenCountCallback(llmTokenCountCallback);