Struct grpc_vision_svc::image_captioning::ImageProcessor

pub struct ImageProcessor { /* private fields */ }

Struct for processing images and generating captions.

Implementations

impl ImageProcessor

pub fn new(models: &Models, device: Device) -> Result<Self>

Creates a new instance of ImageProcessor.

This function initializes the ImageProcessor with the provided models and device. It loads the BLIP and quantized BLIP models, sets up the tokenizer, and prepares the processor for image captioning tasks.

Arguments

• models - A reference to a Models struct containing model configurations.
• device - The device on which the models will be loaded (e.g., CPU or GPU).

Returns

A Result containing the new ImageProcessor instance or an error if initialization fails.

Errors

Returns an error if any of the required models cannot be found or initialized.

pub fn process_image(&self, model: ModelType, image: &[u8]) -> Result<String>

Processes an image and generates a caption.

This function processes the input image using the specified model and generates a textual description of the image. It involves preprocessing the image, converting it into a tensor, passing it through the model to get image embeddings, and then generating text based on these embeddings.

Arguments

• model - The type of model to use for processing the image.
• image - A byte slice containing the image data.

Returns

A Result containing the generated caption as a String or an error if processing fails.

Errors

Returns an error if image processing or caption generation fails.

Trait Implementations

impl Clone for ImageProcessor

fn clone(&self) -> ImageProcessor

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.