# Apple Depth Pro 2 on Mac: Core ML Setup, Speed and License Limits

Start with the small camera model, then check what resolution, frame history and focal length do to your depth map.

By Paras Madan | 2026-11-20


Apple's [Depth Pro 2 research release](https://arxiv.org/abs/2611.26603), submitted November 19, estimates how far scene points sit in front of a single camera. It works with still photos and videos, and Apple provides Core ML packages for a live Mac demo. Start with the small S model at 854 by 480. You can go bigger later, once you know whether the result is sharp and stable enough for your footage.

## Run the camera demo with the S model

Apple's [Core ML model card](https://huggingface.co/apple/DepthPro2-coreml) provides the 51 MB `depthpro2-s_w4.mlpackage`. Clone the [Depth Pro 2 repository](https://github.com/apple-aiml-research/ml-depth-pro-2) and put the package in the repo root. Apple's [Mac demo instructions](https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/demo/README.md) specify Apple silicon, macOS 15 or later, and Xcode 16 or its command-line tools. From the repo root, run:

```bash
./demo/run.sh depthpro2-s_w4.mlpackage
```

macOS may ask for camera access the first time. Start with the built-in camera so you have fewer things to debug. If you're using an external camera that doesn't report focal length, provide the known value with `--focal-px` in pixels at the selected frame size, or put Apple's field-of-view package next to the depth model so the demo can estimate it.

## Check inference speed before increasing resolution

Apple reports a median of 26 ms per prediction at 854 by 480 and 88 ms at 1280 by 720, both with S and a four-frame window on an M4 Max GPU. Those numbers exclude warm-up, preprocessing and rendering. At 30 fps the whole pipeline has about 33 ms to do its work, so a 26 ms inference leaves about 7 ms for everything else. It doesn't guarantee 30 fps in the app, especially on a different Mac.

**A small download can still take several gigabytes to run.** Apple reports roughly 1.5 GB for S at the default resolution and 5 GB at 1280 by 720. If the preview falls behind, test 640 by 360 before moving from the default GPU to the Neural Engine. Apple's docs warn that some larger inputs can fail to compile on the Neural Engine. For a meaningful comparison, time the whole capture-to-display loop after warm-up and count dropped frames.

## Keep state within one camera stream

The pre-exported `w4` packages keep earlier frames in Core ML state. In Swift, create the state once for the stream and pass it to each prediction. Start a fresh state when switching to a new clip or camera. If you create state on every frame, you lose the history that helps prevent depth from jumping around the edge of a moving object.

Try a steady shot of a mug, then move your hand across its edge. If the boundary shifts while the mug stays put, compare the same shot at two resolutions before deciding you need a larger model. For edited clips with hard cuts, process each shot separately so the frames on either side of a cut aren't treated as a continuous scene.

## Compare the numeric depth, not the preview colors

The live model returns inverse depth normalized by focal length. Where that result is positive, distance along the camera's viewing axis in meters is `f_px / (width * inverse_depth)`, using width and focal length in pixels for the frame as fed to the model. A resizing step changes that focal length in pixels. For a saved clip where you need numbers rather than a live overlay, the repository's offline command can export an NPZ with depth already in meters.

For a quick lab check, put a flat object directly in front of the camera at two measured distances. Compare a small patch near the middle of the object instead of a single edge pixel. If both readings drift in the same direction after a resolution change, check the focal length before blaming the checkpoint.

## Check the weights license before product work

Apple's [model license](https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/LICENSE_MODEL) limits use to noncommercial research and specifically excludes product development. The code has separate terms, and the Core ML export doesn't change the model's license. Use the demo to answer a research question about your camera and footage; don't make those weights a dependency of a product you intend to ship under the published terms.

I’m Paras. I work with AI teams on technical writing.
I help turn model releases and integration details into guides developers can use.

[Book a 30-minute call](https://cal.com/paras-madan/30min)
