Interface LlmTokenCountCallback


  • public interface LlmTokenCountCallback
    An interface for calculating the number of tokens used for a given LLM (Large Language Model) and content.

    Implement this interface to define custom logic for token calculation based on your application's requirements.

    Example usage:

    {@code
     class MyTokenCountCallback implements LlmTokenCountCallback {
    • Method Detail

      • calculateLlmTokenCount

        int calculateLlmTokenCount​(java.lang.String model,
                                   java.lang.String content)
        Calculates the number of tokens used for a given LLM model and content.
        Parameters:
        model - The name of the LLM model.
        content - The message content or prompt.
        Returns:
        An integer representing the number of tokens used for the given model and content. If the count cannot be determined or is less than or equal to 0, 0 is returned.