A beginner's guide to the Zoedepth model by Zedge on Replicate

This is a simplified guide to an AI model called Zoedepth maintained by Zedge. If you like these kinds of analysis, you should join AImodels.fyi or follow us on Twitter.

Overview

zoedepth is a monocular depth estimation model that converts single 2D images into depth maps, produced by zedge. The model takes a URI-referenced image as input and outputs a depth map as a URI-accessible file. This is a single-image depth prediction system, meaning it infers 3D spatial information from a photograph without requiring stereo pairs, temporal sequences, or additional sensor data. The model runs on Replicate's infrastructure and accepts a warm_delay parameter for performance tuning during model initialization.

Best use cases

3D reconstruction and scene understanding for robotics. Monocular depth estimation provides essential spatial information for robotic vision systems that need to navigate, grasp, or manipulate objects. The model converts flat camera feeds into metric depth data that downstream systems use for obstacle avoidance, surface planning, and object localization. This is particularly valuable for mobile robots and manipulation systems that cannot rely on stereo cameras or structured light sensors due to size, cost, or environmental constraints.

Augmented reality and virtual object placement. AR applications require accurate depth maps to properly occlude virtual objects behind real-world geometry and place digital content naturally within physical scenes. Rather than using specialized depth sensors, zoedepth generates depth from standard RGB camera input, making it suitable for mobile AR pipelines where depth sensors are unavailable. The output depth map directly enables realistic shadow casting, parallax effects, and collision detection with real-world surfaces.

Cinematic depth-of-field and bokeh simulation. Video production and photo editing workflows use depth maps to apply selective focus effects and synthetic depth-of-field blur that would otherwise require expensive multi-camera rigs or manual masking. Feeding photographs through zoedepth provides the depth information necessary to selectively blur backgrounds while keeping subjects sharp, matching the aesthetic of professional cinematography without requiring specialized hardware.

Autonomous vehicle perception pipelines. Self-driving systems integrate multiple sensor modalities, and monocular depth serves as a complementary signal to LiDAR and radar. zoedepth provides dense depth predictions across the entire image, helping perception stacks detect obstacles, estimate vehicle distances, and build 3D scene representations from camera feeds alone—useful for redundancy, cost reduction, or scenarios where LiDAR is unreliable.

Depth-based image segmentation and scene parsing. Computer vision applications that segment scenes by spatial proximity or layer depth rely on accurate depth maps to distinguish foreground from background. Content moderation systems, semantic segmentation pipelines, and automatic background replacement tools use depth as a primary feature for separating subjects from their surroundings with spatial precision.

Limitations

The model outputs depth as a single URI to an image file rather than structured numerical depth values (like raw arrays or point clouds). This means applications requiring metric depth in specific formats must post-process the output image into numeric arrays, adding an extra decoding step.

Monocular depth estimation has inherent ambiguity at absolute scale—the model cannot determine whether an image shows a small nearby object or a distant large one without additional context. The output is relative depth, not metric depth in physical units, which limits direct use cases requiring real-world measurements without additional calibration or scale references in the image.

The model's accuracy degrades significantly on images with limited visual structure, such as uniform or textureless surfaces, featureless skies, or highly reflective scenes. Images lacking sufficient visual cues for depth inference produce unreliable depth maps with artifacts and hallucinated depth boundaries.

Input images must be valid URIs, requiring image hosting or temporary storage infrastructure rather than direct file uploads. The warm_delay parameter suggests the model benefits from a warm-up period; unoptimized calls may exhibit latency overhead.

The model performs inference on single static images without temporal consistency. For video sequences, running the model frame-by-frame may produce flickering or inconsistent depth changes between frames, requiring temporal smoothing filters as post-processing.

Large images with very high resolutions may encounter memory constraints during inference, though specific resolution limits are not documented in the schema.

How it compares

img2img-combined by zedge is a general-purpose image-to-image model combining multiple transformation capabilities, whereas zoedepth is specialized exclusively for depth estimation. Choose zoedepth when you specifically need spatial geometry; choose img2img-combined when performing style transfer, inpainting, or other non-depth transformations.

deepfashionsdxl by omniedgeio generates high-resolution images from text or conditioning, while zoedepth analyzes existing images to extract depth information. These models serve opposite purposes—generation versus analysis—and would be combined in pipelines where you generate fashion imagery and then need spatial information about the subjects.

zekebooth by zeke is a DreamBooth-based model for personalized image generation and style transfer, whereas zoedepth performs depth extraction from single images. zekebooth creates new images; zoedepth analyzes existing ones for 3D information.

test by anhappdev performs image inpainting—filling missing or masked regions—while zoedepth predicts depth geometry. Depth maps from zoedepth could inform inpainting decisions, but the models target different image transformation problems.

sdxl-deep-down by fofr is a text-to-image diffusion model fine-tuned on underwater imagery, whereas zoedepth estimates depth from existing images. Neither directly competes; they operate at different stages of image processing pipelines.

Technical specifications

The model accepts standard image URIs as input and outputs depth predictions as URI-accessible image files. The OpenAPI schema specifies two inputs: an image parameter (required, string format as URI) and an optional warm_delay integer parameter that controls model initialization behavior.

The model runs on Replicate's containerized infrastructure using Cog version 0.14.1. The latest version was created on April 10, 2025, with version ID fd85428545f04150f59856dab2a51a7be2ca5003a331920b0e4303b17b411332.

Key technical details from the schema:

• Input format: URI-referenced images (HTTP(S) URLs pointing to accessible image files)
• Output format: URI-accessible depth map image file
• warm_delay parameter: Optional integer controlling warm-up behavior; if set, the model returns an empty response after the specified number of seconds, useful for GPU warm-up in batched inference scenarios
• Inference environment: Replicate Cog containerization, version 0.14.1
• Supported image sources: Any publicly accessible image URL or Replicate storage URI

The model does not expose parameter count, training dataset details, or specific architectural information in the available documentation. Depth prediction uses single-image monocular inference without multi-view or temporal processing.

Model inputs and outputs

Inputs

• image (string, URI format, required): URL pointing to the input image that will be analyzed for depth estimation
• warm_delay (integer, optional): Number of seconds to wait during model warm-up; if set, returns empty response after specified delay for performance tuning in batched scenarios

Outputs

• Output (string, URI format): URL pointing to the generated depth map image file, accessible for download or further processing

Getting started

import replicate

client = replicate.Client(api_token="YOUR_REPLICATE_API_TOKEN")

output = client.run(
    "zedge/zoedepth:fd85428545f04150f59856dab2a51a7be2ca5003a331920b0e4303b17b411332",
    input={
        "image": "https://example.com/sample-image.jpg",
        "warm_delay": None
    }
)

print(output)

This returns a URI pointing to the depth map image. Download or process the output image as needed:

import requests
from PIL import Image

depth_map_url = output
response = requests.get(depth_map_url)
depth_image = Image.open(BytesIO(response.content))
depth_image.save("depth_map.png")

Frequently asked questions

Q: What format does the depth map output use?

A: The model outputs the depth map as a URI-accessible image file (typically PNG). You must fetch the URL and decode the image to numeric depth values for downstream processing, as the API returns only the file location, not raw depth arrays.

Q: Can I use this model for real-time video depth estimation?

A: The model processes static images frame-by-frame but does not maintain temporal consistency between frames. For video, you would run inference on each frame individually, which introduces flickering and discontinuities; you should apply temporal smoothing filters as post-processing to stabilize depth predictions across sequences.

Q: Does the model provide metric depth in physical units?

A: No. zoedepth produces relative depth maps without absolute scale information. The output indicates which surfaces are closer or farther but not their precise distance in meters or physical units without additional calibration data or reference objects in the image.

Q: What does the warm_delay parameter do?

A: Setting warm_delay returns an empty response after a specified number of seconds, allowing you to warm up the GPU in batched inference scenarios before processing actual requests. This can reduce latency for subsequent real inference calls by pre-loading the model weights into GPU memory.

Q: Is this model suitable for production applications requiring high accuracy?

A: The model works well for relative depth ordering and scene structure but degrades on textureless surfaces, reflective scenes, and ambiguous geometry. Evaluate accuracy on your specific image domain before production deployment; textureless or uniform regions produce unreliable predictions.

Q: How does monocular depth from this model compare to stereo or LiDAR depth?

A: Monocular depth provides dense predictions across the entire image but lacks absolute scale and struggles with ambiguous or featureless regions. Stereo and LiDAR provide metric depth and better accuracy on textureless surfaces but require specialized hardware; zoedepth works with any RGB camera, making it suitable for software-only systems and as a complementary signal to other depth sensors.

Q: What image resolutions does the model support?

A: The schema does not specify resolution limits. Very high-resolution images may encounter memory constraints during inference on Replicate's infrastructure; test your typical image sizes during development.

Q: Is the model actively maintained?

A: The latest version was updated on April 10, 2025, indicating active maintenance. Check the Replicate model page for version history and any breaking changes between updates.

Click here to read the full guide to Zoedepth