Image Depth Maps Tutorial for iOS: Getting Started

🎓 Courses & Articles
41

Apple does do some processing on the depth data, so there are going to be issues in certain situations around the edges of objects (especially hair!).

There is even more post processing done, if the isDepthDataFiltered property is true. You could try setting it to false and create your own algorithm to fill in the holes in the depth data.

42

Thanks, implementing this algorithm is not something that is possible at the moment due to logistic issues, but maybe the question I asked in SO is a way to solve this? link to the question:

:slight_smile:

43

@toojuice Thank you so much for your replay 
 Yes I make sure it is in this name test10.jpg and it does not work !.. and I am not sure why I only see the top part of my added images in the original mode 
 I tried photos with both direction 
 do you think because I am running this project on mac not iphone? for saving the image could you please give me piece of code 
 unfortunately I have no clue about this

44

Thank you so much for your replay.And I have another question,when I use real machine debugging,after I install this app, I cann’t find it in the Settings,and I don’t know why,hope for your reply.

45

Hi, how are you taking the pictures if you’re not using an iPhone?

46

Hi, do you mean the Settings app on the iPhone? It doesn’t show up there because it doesn’t have any settings and does not require special permissions (i.e. it doesn’t use the camera or have access to the photo library – all photos are packaged with the app)

47

I am taking the photos using iphone X 
 then I send it to the mac to use it in with this code

48

Hi @eyzuky,

Did you try it out? If so, does it look better?

1 Like
49

Hi @seereen2004,

Ok, that should work too. That’s how I created the sample images that come with the tutorial. I have some more questions for you:

  1. Are you using HEIF or converting to JPG?
  2. How are you transferring the images to the Mac? (Airdrop, Image Capture, email, something else?)
  3. When transferring the image, do you have multiple options for the same picture? I.e. do you see two copies of the same image for each portrait-mode shot?
50

It did help! Looks much better :slight_smile:

51

Hi @toojuice
I am using iphone X to capture the portrait images and I used front and back cameras. I tried images with vertical and horizontal positions for the phone
 I found it in JPG formate so I do not have to convert it ! 
 then I send it to the mac using Image Capture 
 and I see 2 images one with E and one without E 
 so I am using images without E to try this app

52

Are all the images that come with the starter/final project still showing up correctly in the Original and Depth mode?

53

Yes your images works perfect
 the images that I captured cannot be shown completely 

try the image in this link 
 this is one of my images that I captured with back camera https://ibb.co/kqZFsH

54

You can try these images as well 


the images with several objects I took them using iphone X
 one with front camera and one with back camera (portrait mode) 
bottle image I took it with iphone 7 plus (portrait mode)
 bottle image I can see just the top part in the app 
 however, the other images I can see just middle part!

55

Ok, so the UIImageView in the view controller is set to Aspect Fill, meaning that it will cut off any extra image on the sides or top that doesn’t match the phone’s aspect ratio. Since the tutorial app was created to only work in portrait mode, you will have to make some adjustments to get this to work with landscape images.

I also took a look at the raw data for your image and compared it to the raw data for one of the included images. It seems like it’s in a different format. The metadata seems to be missing. Within one of the included tutorial images, you can find raw text that looks like XML and some attributes that include the string depthData. This is completely absent from your image.

This makes me think that either 1) no depth data was captured or 2) the depth data was corrupted/removed by some post processing (either on the phone or on your Mac – not sure). Actually, the image sharing service you used may have modified the image, too? Maybe I don’t get the original that way. You could verify by downloading from your link and diffing it with the original.

1 Like
56

OK,I got it.Thank you so much .
There is another problem.Do you know how to save the distance data as a single channel image? I want to use the distance data and RGB image to create its pointcloud file.

57

In the current project, the image for the depth data is created from a CIImage, which is created from a CVPixelBuffer. This will not allow you to write out the data using a function such as UIImagePNGRepresentation or UIImageJPEGRepresentation.

In order to be able to use those function to save the depth data, you will need to first create a CGImage and then create a UIImage from that. When you create a CGImage, you’ll have to be careful with the orientation of the data, so keep that in mind.

58

@toojuice Appreciate your replay 
 is it possible that you post the code that can help us to save the depth ?
Thank you so much

61

This tutorial didn’t discuss capturing photos with depth data programmatically, but you can take a look at this one to see how:

https://www.raywenderlich.com/185885/video-depth-maps-tutorial-for-ios-getting-started

If you want to capture from the TrueDepth camera instead of the Dual camera, just change the AVCaptureDevice from .builtInDualCamera to .builtInTrueDepthCamera.