Class BaseSageMakerContentHandler<InputType, OutputType>Abstract

A handler class to transform input from LLM to a format that SageMaker endpoint expects. Similarily, the class also handles transforming output from the SageMaker endpoint to a format that LLM class expects.

Example:

class ContentHandler implements ContentHandlerBase<string, string> {
  contentType = "application/json"
  accepts = "application/json"

  transformInput(prompt: string, modelKwargs: Record<string, unknown>) {
    const inputString = JSON.stringify({
      prompt,
     ...modelKwargs
    })
    return Buffer.from(inputString)
  }

  transformOutput(output: Uint8Array) {
    const responseJson = JSON.parse(Buffer.from(output).toString("utf-8"))
    return responseJson[0].generated_text
  }

}

Type Parameters

  • InputType

  • OutputType

Hierarchy

  • BaseSageMakerContentHandler

Constructors

constructor

Properties

accepts contentType

Methods

transformInput transformOutput

Constructors

constructor

Properties

accepts

accepts: string = "text/plain"

contentType

contentType: string = "text/plain"

Methods

Abstract transformInput

  • Transforms the prompt and model arguments into a specific format for sending to SageMaker.

    Parameters

    • prompt: InputType

      The prompt to be transformed.

    • modelKwargs: Record<string, unknown>

      Additional arguments.

    Returns Promise<Uint8Array>

    A promise that resolves to the formatted data for sending.

Abstract transformOutput

  • Transforms SageMaker output into a desired format.

    Parameters

    • output: Uint8Array

      The raw output from SageMaker.

    Returns Promise<OutputType>

    A promise that resolves to the transformed data.

Settings

Member Visibility

Theme

On This Page

Generated using TypeDoc