Hi all,
i have a problem in public override void DidUpdateFrame(ARSession session, ARFrame frame) method.
My goal is to draw grid over the frame that arkit give me.
my code:
public override void DidUpdateFrame(ARSession session, ARFrame frame) { using (var s = session) { using (var frm = frame) { try { using (var currentBackgroundFrameImage = s.CurrentFrame?.CapturedImage) { using (var i = Convert22(currentBackgroundFrameImage, frm)) { using (CIContext temporaryContext = CIContext.FromOptions(null)) { CGImage cgImage = temporaryContext.CreateCGImage(i, new CGRect(0, 0, i.Extent.Width, i.Extent.Height)); using (UIImage ni = UIImage.FromImage(cgImage)) { UIGraphics.BeginImageContext(ni.Size); var context = UIGraphics.GetCurrentContext(); ni.Draw(new CGPoint()); context.SetLineWidth(1.0f); context.MoveTo(0, 0); context.AddLineToPoint(ni.Size.Width, ni.Size.Height / 2); context.SetStrokeColor(new CGColor(255, 0, 0)); context.StrokePath(); var newImage = UIGraphics.GetImageFromCurrentImageContext(); UIGraphics.EndImageContext(); this.control.SceneView.Scene.Background.Contents = newImage; } } } } } catch (Exception e) { Debug.WriteLine(e.Message); } } } }
Well, my convert method is used to create and apply transform to crop and scale correctly the image from PixelBuffer:
public CIImage Convert22(CVPixelBuffer pixelBuffer, ARFrame frame) { var imageSize = new CGSize(pixelBuffer.Width, pixelBuffer.Height); var sizT = this.control.SceneView.CurrentViewport; var viewport = this.control.SceneView.Bounds; var viewportSize = viewport.Size; var interfaceOrientation = this.control.SceneView.Window.WindowScene.InterfaceOrientation; var image = new CIImage(pixelBuffer); var normalizeTransform = CGAffineTransform.MakeScale(1.0f / imageSize.Width, 1.0f / imageSize.Height); var portraitScale = CGAffineTransform.MakeScale(-1.0f, -1.0f); portraitScale = CGAffineTransform.Translate(portraitScale, -1, -1); var flipTransform = interfaceOrientation.IsPortrait() ? portraitScale : CGAffineTransform.MakeIdentity(); var displayTransform = frame.GetDisplayTransform(interfaceOrientation, viewportSize); var toViewPortTransform = CGAffineTransform.MakeScale(viewportSize.Width, viewportSize.Height); var finalT = normalizeTransform * flipTransform * displayTransform * toViewPortTransform; return image.ImageByApplyingTransform(finalT).ImageByCroppingToRect(viewport); }
Application run...but after few seconds is blocked.. Why?
I have found into the Iphone log this "Error":
this error appears when i try to use pixel buffer data into ** var image = new CIImage(pixelBuffer);** but no exception handled....
Personally i don't know how to solve problem.
CIImage is basic to apply transform to obtain the correct UIImage cropped for Viewport.
Thanks.