PHP Prisma

Light-weight PHP package for integrating multi-media related Large Language Models (LLMs) into your applications using a unified interface.

Supported providers

Images

  background describe detext erase imagine inpaint isolate recognize relocate repaint uncrop upscale vectorize
Bedrock Titan - - - - yes yes yes - - - - - yes
Black Forest Labs - - - - beta beta - - - - beta - -
Clipdrop yes - yes yes yes - yes - - - yes yes -
Cohere - - - - - - - - - - - - yes
Gemini - yes - - yes - - - - yes - - -
Ideogram beta beta - - beta beta - - - beta - beta -
Mistral - - - - - - - yes - - - - -
OpenAI - yes - - yes yes - - - - - - -
RemoveBG - - - - - - yes - yes - - - -
StabilityAI - - - yes yes yes yes - - - yes yes -
VertexAI - - - - yes yes - - - - - yes yes
VoyageAI - - - - - - - - - - - - yes

Installation

composer req aimeos/prisma

API usage

Basic usage:

use Aimeos\Prisma\Prisma;

$image = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->model( '<modelname>' ) // if model can be selected
    ->ensure( 'imagine' ) // make sure interface is implemented
    ->imagine( 'a grumpy cat' )
    ->binary();

ensure

Ensures that the provider has implemented the method.

public function ensure( string $method ) : self

Example:

\Aimeos\Prisma\Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->ensure( 'imagine' );

has

Tests if the provider has implemented the method.

public function has( string $method ) : bool

Example:

\Aimeos\Prisma\Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->has( 'imagine' );

model

Use the model passed by its name.

Used if the provider supports more than one model and allows to select between the different models. Otherwise, it’s ignored.

public function model( ?string $model ) : self

Example:

\Aimeos\Prisma\Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->model( 'dall-e-3' );

withClientOptions

Add options for the Guzzle HTTP client.

public function withClientOptions( array `$options` ) : self

Example:

\Aimeos\Prisma\Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->withClientOptions( ['timeout' => 120] );

withSystemPrompt

Add a system prompt for the LLM.

It may be used by providers supporting system prompts. Otherwise, it’s ignored.

public function withSystemPrompt( ?string $prompt ) : self

Example:

\Aimeos\Prisma\Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->withSystemPrompt( 'You are a professional illustrator' );

Response objects

The methods return a FileResponse, TextResponse or VectorResponse object that contains the returned data with optional meta/usage/description information.

FileResponse objects:

$base64 = $response->base64(); // from binary, base64 and URL, waits for async requests
$file = $response->binary(); // from binary, base64 and URL, waits for async requests
$url = $response->url(); // only if URL is returned, otherwise NULL
$mime = $response->mimetype(); // image mime type, waits for async requests
$text = $response->description(); // image description if returned by provider
$bool = $response->ready(); // False for async APIs until file is available

URLs are automatically converted to binary and base64 data if requested and conversion between binary and base64 data is done on request too.

TextResponse objects:

$text = $response->text(); // text content (non-streaming)

VectorResponse objects:

$vectors = $response->vectors(); // embedding vectors for the passed files in the same order
$vector = $response->first(); // first embedding vector if only one file has been passed

Included meta data (optional):

$meta = $response->meta();

It returns an associative array whose content totally depends on the provider.

Included usage data (optional):

$usage = $response->usage();

It returns an associative array whose content depends on the provider. If the provider returns usage information, the used array key is available and contains a number. What the number represents depdends on the provider too.

Image API

Most methods require an image object as input which contains a reference to the image that should be processed. This object can be created by:

use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.php', 'image/png' );
$image = Image::fromLocalPath( 'path/to/image.png', 'image/png' );
$image = Image::fromBinary( 'PNG...', 'image/png' );
$image = Image::fromBase64( 'UE5H...', 'image/png' );

// Laravel only:
$image = Image::fromStoragePath( 'path/to/image.png', 'public', 'image/png' );

The last parameter of all methods (mime type) is optional. If it’s not passed, the file content will be retrieved to determine the mime type if reqested.

Note: It’s best to use fromUrl() if possible because all other formats (binary and base64) can be derived from the URL content but URLs can’t be created from binary/base64 data.

background

Replace image background with a background described by the prompt.

public function background( Image $image, string $prompt, array $options = [] ) : FileResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->background( $image, 'Golden sunset on a caribbean beach' );

$image = $fileResponse->binary();

describe

Describe the content of an image.

public function describe( Image $image, ?string $lang = null, array $options = [] ) : TextResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );

$textResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->describe( $image, 'de' );

$text = $textResponse->text();

detext

Remove all text from the image.

public function detext( Image $image, array $options = [] ) : FileResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->detext( `$image` );

$image = $fileResponse->binary();

erase

Erase parts of the image.

public function erase( Image $image, Image $mask, array $options = [] ) : FileResponse

The mask must be an image with black parts (#000000) to keep and white parts (#FFFFFF) to remove.

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );
$mask = Image::fromBinary( 'PNG...' );

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->erase( $image, $mask );

$image = $fileResponse->binary();

imagine

Generate an image from the prompt.

public function imagine( string $prompt, array $images = [], array $options = [] ) : FileResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->imagine( 'Futuristic robot looking at a dashboard' );

$image = $fileResponse->binary();

inpaint

Edit an image by inpainting an area defined by a mask according to a prompt.

public function inpaint( Image $image, Image $mask, string $prompt, array $options = [] ) : FileResponse

The mask must be an image with black parts (#000000) to keep and white parts (#FFFFFF) to edit.

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );
$mask = Image::fromBinary( 'PNG...' );

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->inpaint( $image, $mask, 'add a pink flamingo' );

$image = $fileResponse->binary();

isolate

Remove the image background.

public function isolate( Image $image, array $options = [] ) : FileResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->isolate( `$image` );

$image = $fileResponse->binary();

recognize

Recognizes the text in the given image (OCR).

public function recognize( Image $image, array $options = [] ) : TextResponse;

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );

$textTesponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->recognize( `$image` );

$text = $textResponse->text();

relocate

Place the foreground object on a new background.

public function relocate( Image $image, Image $bgimage, array $options = [] ) : FileResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );
$bgimage = Image::fromUrl( 'https://example.com/background.png' );

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->relocate( $image, $bgimage );

$image = $fileResponse->binary();

repaint

Repaint an image according to the prompt.

public function repaint( Image $image, string $prompt, array $options = [] ) : FileResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->repaint( $image, 'Use a van Goch style' );

$image = $fileResponse->binary();

uncrop

Extend/outpaint the image.

public function uncrop( Image $image,  int $top, int $right, int $bottom, int $left, array $options = [] ) : FileResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->uncrop( $image, 100, 200, 0, 50 );

$image = $fileResponse->binary();

upscale

Scale up the image.

public function upscale( Image $image, int $factor, array $options = [] ) : FileResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$image = Image::fromUrl( 'https://example.com/image.png' );

$fileResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->upscale( $image, 4 );

$image = $fileResponse->binary();

vectorize

Creates embedding vectors of the images’ content.

public function vectorize( array $images, ?int $size = null, array $options = [] ) : VectorResponse

Supported options:

Example:

use Aimeos\Prisma\Prisma;
use \Aimeos\Prisma\Files\Image;

$images = [
    Image::fromUrl( 'https://example.com/image.png' ),
    Image::fromUrl( 'https://example.com/image2.png' ),
];

$vectorResponse = Prisma::image()
    ->using( '<provider>', ['api_key' => 'xxx'])
    ->vectorize( $images, 512 );

$vectors = $vectorResponse->vectors();

Custom providers

Image provider

To create a custom Prisma image provider, use this skeleton and implement all Prisma interfaces supported by the remote API:

<?php

namespace Aimeos\Prisma\Providers\Image;

use Aimeos\Prisma\Contracts\Image\Imagine;
use Aimeos\Prisma\Exceptions\PrismaException;
use Aimeos\Prisma\Files\Image;
use Aimeos\Prisma\Providers\Base;
use Aimeos\Prisma\Responses\FileResponse;
use Psr\Http\Message\ResponseInterface;


class Myprovider extends Base implements Imagine
{
    public function __construct( array $config )
    {
        if( !isset( $config['api_key'] ) ) {
            throw new PrismaException( sprintf( 'No API key' ) );
        }

        // if authentication is done via headers
        $this->header( '<api key name>', $config['api_key'] );
        // base url for all requests (no paths)
        $this->baseUrl( '<provider URL>' );
    }


    public function imagine( string $prompt, array $images = [], array $options = [] ) : FileResponse
    {
        // filter key/value pairs in $options and use the ones allowed by the API
        $allowed = $this->allow( $options, ['<key1>', '<key2>', /* ... */] );
        // filter values to pass only allowed option values
        $allowed = $this->sanitize( $allowed, ['<key1>' => ['<val1>', '<val2>', '<val3>']])

        // Form data
        $data = $this->request( allowed );
        // Multipart data
        $data = ['multipart' => $this->request( allowed, ['image_key' => $image->binary()] )];
        // JSON data
        $data = ['json' => ['image_key' => $image->base64()] + allowed];

        // use Guzzle to send the request and get the response from the server
        $response = $this->client()->post( 'relative/path', $data );
        return $this->toFileResponse( $response );
    }


    protected function toFileResponse( ResponseInterface $response ) : FileResponse
    {
        // from Base class, overwrite as needed
        $this->validate( $response );

        // use binary content or decode JSON content
        $content = $response->getBody()->getContents();

        // if mime type is available in header
        $mimetype = $response->getHeaderLine( 'Content-Type' );

        // use fromBinary(), fromBase64() or fromUrl()
        return FileResponse::fromBinary( content, mimetype )
            ->withDescription( // optional
                '' // image description if returned
            )
            ->withUsage( // optional
                100, // used tokens, credits, etc. if available or NULL
                [] // key/value pairs for the rest of the usage data
            )
            ->withMeta( // optional
                [] // meta data as key/value pairs
            );
    }