Всё работает в алпайне, показывайте содержимое Dockerfile или пишет подробнее что не так.
docker run --rm -it alpine sh -c "apk update && apk add build-base imagemagick-dev imagemagick && convert"
вывод:
Version: ImageMagick 7.1.0-50 beta Q16-HDRI x86_64 20489 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules
Delegates (built-in): bzlib cairo fontconfig freetype gslib heic jng jpeg jxl lcms ltdl lzma png ps rsvg tiff webp x xml zlib
Compiler: gcc (11.2)
Usage: convert [options ...] file [ [options ...] file ...] [options ...] file
Image Settings:
-adjoin join images into a single multi-image file
-affine matrix affine transform matrix
-alpha option activate, deactivate, reset, or set the alpha channel
-antialias remove pixel-aliasing
-authenticate password
decipher image with this password
-attenuate value lessen (or intensify) when adding noise to an image
-background color background color
-bias value add bias when convolving an image
-black-point-compensation
