Configuring object detection using SAM2 with Kdenlive on a Mac

Even if you don't use Kdenlive for your everyday video editing, you might have seen their neat demo of the new SAM2 integration from Facebook.

The Kdenlive team showed off the demo on Mastodon

Having rushed to try this on your Mac, you might have run into a couple of issues getting it running! I couldn't find anyone else who had documented getting it to run, so here are the steps for anyone wanting to try it!

Ensure you have a modern version of python3 installed

The system version of Python3 is 3.9, which is not modern enought to run SAM2. You will see the following error when trying to install the plugin:

Collecting opencv-python (from -r /Applications/kdenlive.app/Contents/Resources/scripts/automask/requirements-sam.txt (line 4))
Downloading opencv_python-4.11.0.86-cp37-abi3-macosx_13_0_arm64.whl.metadata (20 kB)
INFO: pip is looking at multiple versions of sam-2 to determine which version is compatible with other requirements. This could take a while.
ERROR: Package 'sam-2' requires a different Python: 3.9.6 not in '>=3.10.0'
failed installing /Applications/kdenlive.app/Contents/Resources/scripts/automask/requirements-sam.txt

Make sure you have installed a newer version either using brew or from the python website.

Run kdenlive from the terminal

To make sure kdenlive actually uses the newer version of python, run it directly from the terminal, after validating that the version of python is up-to-date!

python3 --version
/Applications/kdenlive.app/Contents/MacOS/kdenlive

At this point when you try to install the plugin, you might run into the following error:

kf.kio.core: couldn't create worker: "Can not find 'kioworker' executable at '/Applications/kdenlive.app/Contents/MacOS, /Applications/kdenlive.app/Contents/libexec, /Applications/kdenlive.app/Contents/libexec/kf6, /Applications/kdenlive.app/Contents/Frameworks/libexec/kf6, /Users/gitlab/ws/builds/GZwHuM5x/0/sysadmin/ci-management/macos-arm-clang/lib/libexec/kf6'"
kf.kio.workers.file: readData() returned -1

This is because on mac we don't have the KDE framework installed, so the final step is to download the model directly from the Facebook repo and copy it into the kdenlive model folder at /Users/user/Library/Application\ Support/kdenlive/sam2models/sam2.1_hiera_tiny.pt. Change the username, and model name as appropriate.

Finally you should have smooth object detection!

SAM2 Plugin correctly configured

Object mask successfully created on Mac