Is there a correct way to save the depth map to an image?

Question

I'm using ARKit to collect LiDAR data. I have read the depth map and its format is 'kCVPixelFormatType_DepthFloat32'. I saved the depth map to a PNG image by converting it to UIImage, but I found that the PNG depth map is incorrect. The PNG format only supports 16bit data.

let ciImage = CIImage(cvPixelBuffer: pixelBuf)
let cgImage = context.createCGImage(ciImage, from: ciImage.extent)
let uiImage = UIImage(cgImage: cgImage!).pngData()

So, I have to save the depth map to a TIFF image.

let ciImage = CIImage(cvPixelBuffer: pixelBuf)
       
do {
    try context.writeTIFFRepresentation(of: ciImage, to: path, format: context.workingFormat, colorSpace: context.workingColorSpace!, options: [:])
} catch {
    self.showInfo += "Save TIFF failed;"
    print(error)
}

How to convert the depth map from kCVPixelFormatType_DepthFloat32 to Float16? Is there a correct way to save the depth map to a PNG image?

2k

Posted by

lawpdas

Reply

Add a Comment

Answer 1

Hi, were you ever able to answer this question? I have the same problem right now.

Thanks in advance!

Posted by

M_Bischof

For future reference, I think the correct way can be wound in this example project by Apple: https://developer.apple.com/documentation/realitykit/taking_pictures_for_3d_object_capture

let depthData = photo.depthData?.converting(toDepthDataType: kCVPixelFormatType_DisparityFloat32)

This can be used with kCVPixelFormatType_DisparityFloat16 / kCVPixelFormatType_DisparityFloat32 / kCVPixelFormatType_DepthFloat16 / kCVPixelFormatType_DepthFloat32

—
M_Bischof

Add a Comment

Answer 2

 // Auxiliary function to make String from depth map array
    func getStringFrom2DimArray(array: [[Float32]], height: Int, width: Int) -> String
    {
        var arrayStr: String = ""
        for y in 0...height - 1
        {
            var lineStr = ""
            for x in 0...width - 1
            {
                lineStr += String(array[y][x])
                if x != width - 1
                {
                    lineStr += ","
                }
            }
            lineStr += "\n"
            arrayStr += lineStr
        }
        return arrayStr
    }

 let depthWidth2 = CVPixelBufferGetWidth(depthMap)
        let depthHeight2 = CVPixelBufferGetHeight(depthMap)
        CVPixelBufferLockBaseAddress(depthMap, CVPixelBufferLockFlags(rawValue: 0))
        let floatBuffer = unsafeBitCast(CVPixelBufferGetBaseAddress(depthMap), to: UnsafeMutablePointer<Float32>.self)
        var depthArray = [[Float32]]()
        for y in 0...depthHeight2 - 1
        {
            var distancesLine = [Float32]()
            for x in 0...depthWidth2 - 1
            {
                let distanceAtXYPoint = floatBuffer[y * depthWidth2 + x]
                distancesLine.append(distanceAtXYPoint)
                print("Depth in (\(x), \(y)): \(distanceAtXYPoint)")
            }
            depthArray.append(distancesLine)
        }

        let textDepthUrl = documentsDirectory.appendingPathComponent("depth_text_\(dateString).txt")
        
        let depthString: String = getStringFrom2DimArray(array: depthArray, height: depthHeight2, width: depthWidth2)

 try depthString.write(to: textDepthUrl, atomically: false, encoding: .utf8)

Posted by

vaneesh007

Add a Comment

Is there a correct way to save the depth map to an image?

Replies