[Proposal / discussion] Optional libvips backend for webp_transform

Start with ImageTransform.cc and the proposed CMake builds, then review the libvips and ImageMagick backend trade-offs described in the issue. Done means reaching a decision on whether to pursue libvips, which plugin form to use, and how to handle licensing, RSS behavior, and the experimental/magick relationship.

This is a proposal for discussion, not a request to merge. It summarizes an evaluation of libvips as an alternative to ImageMagick for webp_transform, including trade-offs that argue against switching. Input from the community is wanted on whether to pursue it and in what form.

Motivation: attack surface

webp_transform decodes untrusted origin bodies on ET_NET threads. The case for libvips is security, not speed.

Advisory history, counted on the same basis for both libraries:

source	ImageMagick	libvips
NVD keyword match	about 860	about 29 (many assigned by VulDB in 2026)
GitHub security advisories	the 100 most recent span 2026-05-30 to 2026-09-27	12 in total, since 2023

ImageMagick:

• Several recent advisories appear, from their descriptions, to be reachable through plain JPEG/PNG/WebP reads that a security policy cannot block. This has not been verified against the plugin's code path.
  • XMP profile parsing: DoS, infinite loop, use-after-free, over-read
  • JPEG decoder information disclosure
  • 8BIM use-after-free (identify path)
• Image::read() sniffs the format itself, so a coder allowlist depends on a correct policy.xml.

libvips:

• In a build with only jpeg/png/webp/exif enabled, one of the 12 GitHub advisories is reachable: an EXIF NULL dereference, fixed in 8.18.2.
• The rest are in tiff, heif, svg, pdf, gif, radiance, ppm-source, the vips native format, and conversion ops the plugin doesn't call.
• The plugin already knows the format from its signature check, so it can call vips_jpegload_buffer / vips_pngload_buffer / vips_webpload_buffer directly. libvips never sniffs.
• vips_operation_block_set() can block every other loader as defence in depth.
• New third-party parsers come with it: libexif (24 NVD entries, 4 in 2026; optional, though disabling it drops EXIF passthrough), lcms2 (needed only for CMYK JPEGs) and GLib.

Not a motivation: speed or output size

• Throughput and CPU are a tie. libjpeg and libwebp do the work in both.
  • At matched quality, standalone per-image latency is within about ±10% across image classes (measured against ImageMagick 7.1.0-1).
  • Inside traffic_server with 16 clients, throughput is the same: 88–100 rps for ImageMagick 7.1.2-32 vs 92 rps for libvips, on 8 vCPU.
• Output size at equal quality matches within 1–4%.

Memory: better per image, worse per process (open risk)

Peak memory per transcode is 28–48% lower with libvips. ImageMagick 7 (Q16 HDRI) holds pixels as floats.

image	ImageMagick	libvips
1920x1277 JPEG -> WebP	55 MB	29 MB

Inside traffic_server, process RSS is worse with libvips (jemalloc linked, default settings, 8 ET_NET threads, sustained transform load):

setup	mean RSS
ImageMagick	411 MB
libvips	631 MB
libvips + jemalloc narenas:8	344 MB
ImageMagick + jemalloc narenas:8	351 MB

• Cause: libvips does pixel work on its own worker threads. jemalloc's default of 4 arenas per CPU then keeps freed memory in many arenas. Allocator tuning closes the gap, but that is a process-wide setting.
• Unknown: behaviour on hosts with many more cores.
• Hazard: capping libvips' thread pool (VIPS_MAX_THREADS=8) hung traffic_server under load. The cause wasn't confirmed (no thread dump was taken). The plugin wouldn't set it, but it is a sharp edge.

Licensing

• libvips and GLib are LGPL-2.1+, Category X under ASF policy.
• A libvips backend can only be an optional, separately installed dependency, never bundled: "the component is only needed for optional features" (https://www.apache.org/legal/resolved.html).
• webp_transform is already built only when its image library is found.

What a prototype looks like

• Interface: a small Transcoder interface (init(), transcode()) with ImageMagick and libvips implementations. ImageTransform.cc keeps all HTTP logic.
• Builds: CMake builds webp_transform.so (ImageMagick) and/or webp_transform_vips.so (libvips >= 8.13 via pkg-config).
• libvips settings:
  • vips_concurrency_set(1)
  • vips_cache_set_max(0)
  • vips_block_untrusted_set(TRUE)
  • all loaders blocked except the three buffer loaders
  • sequential access with fail_on=error
  • dimensions checked from the header before any pixels are decoded
• Behaviour matches the ImageMagick backend on the edge cases tested:
  • oversized, truncated and mislabeled bodies pass through
  • CMYK converts with lcms2 (colours differ slightly for CMYK files without an embedded profile)
  • EXIF orientation is preserved
• Differences to account for:
  • Error log strings differ. webp_transform_decode_limit matches ImageMagick.. error.
  • libvips writes an EXIF block, so its JPEGs have no JFIF APP0 marker. webp_transform_invalid_input checks for JFIF.
  • Output quality must be explicit, because libvips has no source-quality estimate. This fits the quality proposal in #13772.

Related: experimental/magick

plugins/experimental/magick runs ImageMagick command lines (MagickCommandGenesis), which can't be ported to libvips. A libvips webp_transform would not remove ImageMagick from builds that enable experimental plugins.

Questions for discussion

1. Is reduced attack surface worth an LGPL optional dependency (libvips + GLib) and a second backend to maintain?
2. If yes, should libvips be an alternative build of the same plugin, a separate plugin, or eventually the only backend?
3. Can we accept the RSS behaviour under default jemalloc settings, or would we need guidance or tuning? Data from larger hosts would help.
4. Should experimental/magick stay as is, independent of this?

Method

• Host: 8 vCPU (Ice Lake), EL9, gcc 11.
• Libraries: ImageMagick 7.1.2-32 (Q16 HDRI, OpenMP, open policy; standalone latency also on 7.1.0-1); libvips 8.18.7 (meson, jpeg/png/webp/exif/lcms2 only).
• Codecs: libwebp 1.2.0, libpng 1.6.37, libjpeg-turbo 2.0.90.
• Corpus:
  • 36 web JPEGs (500–1920 px)
  • 24 Kodak PNGs
  • 8 RGBA logos
  • 28 WebP
  • edge cases
• Test: traffic_server master (RelWithDebInfo, jemalloc 5.2.1), cache off for load runs, 16 clients for 60 s, RSS sampled every second.
• Harness and full data available on request.