Is JPEG compression SIMD-optimized?
libjpeg-turbo replacement for libjpeg is SIMD-optimized. In order to get Pillow or its faster fork Pillow-SIMD to use libjpeg-turbo, the latter needs to be already installed during the former’s compilation time. Once Pillow is compiled/installed, it no longer matters which libjpeg version is installed in your virtual environment or system-wide, as long as the same libjpeg library remains at the same location as it was during the compilation time (it’s dynamically linked).
However, if at a later time something triggers a conda or pip update on Pillow it will fetch a pre-compiled version which most likely is not built against libjpeg-turbo and replace your custom built Pillow or Pillow-SIMD.
Here is how you can see that the PIL library is dynamically linked to libjpeg.so:
cd ~/anaconda3/envs/fastai/lib/python3.6/site-packages/PIL/
ldd _imaging.cpython-36m-x86_64-linux-gnu.so | grep libjpeg
libjpeg.so.8 => ~/anaconda3/envs/fastai/lib/libjpeg.so.8
and ~/anaconda3/envs/fastai/lib/libjpeg.so.8 was installed by conda install -c conda-forge libjpeg-turbo. We know that from:
cd ~/anaconda3/envs/fastai/conda-meta/
grep libjpeg.so libjpeg-turbo-2.0.1-h470a237_0.json
If I now install the normal libjpeg and do the same check on the jpeg’s package info:
conda install jpeg
cd ~/anaconda3/envs/fastai/conda-meta/
grep libjpeg.so jpeg-9b-h024ee3a_2.json
I find that it’s lib/libjpeg.so.9.2.0 (~/anaconda3/envs/fastai/lib/libjpeg.so.9.2.0).
Also, if libjpeg-turbo and libjpeg happen to have the same version number, even if you built Pillow or Pillow-SIMD against libjpeg-turbo, but then later replaced it with the default jpeg with exactly the same version you will end up with the slower version, since the linking happens at build time. But so far that risk appears to be small, as of this writing, libjpeg-turbo releases are in the 8.x versions, whereas jpeg’s are in 9.x’s.
How to tell whether Pillow or Pillow-SIMD is using libjpeg-turbo?
You need Pillow>=5.4.0 to accomplish the following (install from github until then: pip install git+
https://github.com/python-pillow/Pillow).
python -c "from PIL import features; print(features.check_feature('libjpeg_turbo'))"
True
And a version-proof check:
from PIL import features, Image
from packaging import version
try: ver = Image.__version__ # PIL >= 7
except: ver = Image.PILLOW_VERSION # PIL < 7
if version.parse(ver) >= version.parse("5.4.0"):
if features.check_feature('libjpeg_turbo'):
print("libjpeg-turbo is on")
else:
print("libjpeg-turbo is not on")
else:
print(f"libjpeg-turbo' status can't be derived - need Pillow(-SIMD)? >= 5.4.0 to tell, current version {ver}")
