Add a filter (or setting?) to allow alt text generation when the site has basic authentication

Open
#895 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
32/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
php
Domain
ai, backend

Research direction

Start at the classifai_openai_api_request_post_options filter and trace the alt-text request path for classifai_feature_descriptive_text_generator. Compare the existing request options with the supplied basic-auth workaround; done should mean images from basic-authenticated sites can be processed without requiring the provider to fetch them directly.

Written by the indexing model from the issue text.

Description

Is your enhancement related to a problem? Please describe.

Right now alt text generation fails when basic authentication is enabled because OpenAI (or whichever provider) can't access the image directly.

This code works specifically for my use-case, but I'm wondering if there is a way to more easily handle this while we're working on protected staging sites.

/**
 * Handle basic auth for Classifai alt text generation via OpenAI.
 *
 * @param array  $options The options.
 * @param string $url     The URL.
 * @param string $feature The feature.
 *
 * @return array
 */
add_filter( 'classifai_openai_api_request_post_options', function( array $options, string $url, string $feature ): array {
	// Bail if not our feature.
	if ( 'classifai_feature_descriptive_text_generator' !== $feature ) {
		return $options;
	}

	// Bail if no body.
	if ( ! isset( $options['body'] ) ) {
		return $options;
	}

	// Bail if no messages.
	$body = json_decode( $options['body'], true );
	if ( ! $body || empty( $body['messages'] ) ) {
		return $options;
	}

	// Set the username and password.
	$un = 'someun';
	$pw = 'somepw';

	// Encode the credentials.
	$creds = base64_encode( $un . ':' . $pw );

	// Loop through the messages.
	foreach ( $body['messages'] as &$message ) {
		// Bail if not an array.
		if ( ! is_array( $message['content'] ) ) {
			continue;
		}

		// Loop through the items.
		foreach ( $message['content'] as &$item ) {
			// Bail if not an image.
			if ( empty( $item['type'] ) || 'image_url' !== $item['type'] ) {
				continue;
			}

			// Fetch the image manually.
			$resp = wp_remote_get( $item['image_url']['url'], [
				'headers' => [ 'Authorization' => 'Basic ' . $creds ],
			] );

			// Bail if error.
			if ( is_wp_error( $resp ) ) {
				continue;
			}

			// Get the raw body.
			$raw  = wp_remote_retrieve_body( $resp );

			// Encode the body.
			$b64  = base64_encode( $raw );

			// Figure out the MIME type (fallback to jpeg).
			$info = @getimagesizefromstring( $raw );
			$mime = $info ? $info['mime'] : 'image/jpeg';

			// Replace with a Data URI.
			$item = [
				'type'      => 'image_url',
				'image_url' => [
					'url'    => "data:{$mime};base64,{$b64}",
					'detail' => $item['image_url']['detail'] ?? 'auto',
				],
			];
		}
	}

	// Encode the body.
	$options['body'] = wp_json_encode( $body );

	return $options;

}, 10, 3  );
Designs

No response

Describe alternatives you've considered

No response

Code of Conduct
  • I agree to follow this project's Code of Conduct
Dominant language
PHP
Stars
714
Forks
68
Avg merge
4h 32m
Merged PRs (30d)
23

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from 10up/classifai

All issues in 10up/classifai

Similar issues

More PHP issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.