Resize View on content changed

I have the following situation:

On my xaml page I have a setup that looks like this:

Open Bracket ScrollView Close Bracket
Open Bracket local:MyView x:Name="Test" BackgroundColor="White" BackgroundImageFilePath="{Binding BackgroundAttachment.LocalAttachmentFilePath}"
StrokeColor="{Binding StrokeColor}" StrokeWidth="{Binding StrokeWidth}"/ Close Bracket
Open Bracket /ScrollView Close Bracket

This is, roughly, supposed to be a drawing pad. the user can set a background image and then draw all over it.

It has a Renderer, which contains these fields:

    ImageView imageView;
    Canvas sketchCanvas;
    Bitmap backgroundBmp; 

It has an OnElementChanged which does roughly:

protected override void OnElementChanged(ElementChangedEventArgs<SketchPadView> e)
        {
            base.OnElementChanged(e);
            if (Control == null && e.NewElement != null)
            {

                imageView = new Android.Widget.ImageView(Forms.Context);
                //imageView.SetMinimumHeight(2300);
                //imageView.SetMinimumWidth(2300);
                SetNativeControl(imageView);                
            }
            if(e.OldElement != null)
            {
                e.NewElement.SketchBytesRequested -= OnSketchBytesRequested;                
            }
            if (e.NewElement != null)
            {
                e.NewElement.SketchBytesRequested += OnSketchBytesRequested;
                Element.Reset = () => Reset();
                Element.Redo = () => Redo();
                Element.Undo = () => Undo();                
            }
        }

and then there are other methods that set sketchCanvas and backgroundBmp into imageView.

The problem comes when zooming.

I'm doing:

private void RescaleImage(float newParentScaling)
        {
            float oldParentScaling = ParentScaling;
            float removeOldParentScaling = 1 / oldParentScaling;
            ParentScaling = newParentScaling;
            sketchCanvas.Scale(removeOldParentScaling, removeOldParentScaling);
            sketchCanvas.Scale(newParentScaling, newParentScaling);
            foreach (var pInfo in sketchPathInfos)
            {
                Matrix m = new Matrix();
                m.SetScale(removeOldParentScaling, removeOldParentScaling);
                pInfo.Path.Transform(m);
                m.SetScale(newParentScaling, newParentScaling);
                pInfo.Path.Transform(m);
            }
        }

Which works well enough for handling the sketchCanvas, and backgroundBmp is also taken care of. But the imageView size doesn't change. I can give it a height/width in its constructor (up in onElementChanged), and if I make them very big there then my scrollView (all the way at the top in the xamarin page) dutifully lets me scroll up and down it. But most of it starts out as empty. That is, I can scroll around a big empty imageView with a small sketchCanvas up there in the corner.

What i'd like to do is change the size of my imageView along with my sketchCanvas, such that the scrollView would show a scrollbar when imageView got big, and not do so when it was small. In a perfect world it would also start out the same size as the scrollView, a la 'FillAndExpand' behavior.

But the only thing I can seem to set on the imageView is the minimumHeight/minimumWidth parameters, and while those affect the ScrollView if I set them at the beginning, they don't seem to do so if I set them at the end of my RescaleImage function. Is there a way to set the imageView's height/width, or in other ways resize it, such that I can have MyView grow and shrink with its content, with the user scrolling about in the scrollView when it is too large?