Porting Google WebRTC app to C# - Android (and iOS in the future, MVVMCross) - native lib problem

Hi

Edit: Attempting to fix markup

Main machine setup *Unbuntu for building the WebRTC code *Window 7 and Visual Studio 2013 with Xamarin.iOS and Xamarin.Android for building apps

Background: I wish to build a cross platform app that supports WebRTC (real time communication via video/audio). Currently this is in the starting stages, and if I manage to get this developed further then this will be included in a main app. For now I am just working on porting and implementing the Java WebRTC demo app in C#.

• http://www.webrtc.org/
• http://en.wikipedia.org/wiki/WebRTC

Current stage: I have managed to port the Java demo app to c# so it builds, there are very likely some conversion errors and problems, but the app builds with no errors. I am now stuck at getting marshaling interop exceptions when trying to invoke the JNI enabled methods in the .so file I built following

http://devryazantsev.blogspot.ru/p/building-webrtc-trunk-for-andoid-and.html

After following this blogposting, and copying the resulting files to my windows machine you end up with

libwebrtc-video-demo-jni.so (located at: webrtc\video_engine\test\android\libs\armeabi-v7a)

main app - java files
1. webrtc\video_engine\test\android\src\org\webrtc\videoengineapp

supporting classes - java files
1. webrtc\modules\audio_device\android\java\src\org\webrtc\voiceengine
2. webrtc\modules\video_capture\android\java\src\org\webrtc\videoengine
3. webrtc\modules\video_render\android\java\src\org\webrtc\videoengine

I have converted the Java files to C# equivalents (or at least I think I have -> it compiles)

Initially I had some problems using DllImport since it could not find the methods in the .so file but I found that method names were scrambled, which I found by using this Android tools command on my Ubuntu build machine

/WebRTCDemo/trunk/webrtc/video_engine/test/android/libs/armeabi-v7a$ arm-linux-androideabi-nm -D libwebrtc-video-demo-jni.so

which yielded something like

00015358 T JNI_OnLoad
00015be4 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_AddRemoteRenderer
00016e3c T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_CreateChannel
00015f14 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_EnableNACK
00015f58 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_EnablePLI
00015e04 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetCameraOrientation
00015acc T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetCodecs
000153d4 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetVideoEngine
000154cc T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_Init
000153d0 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit
00015c30 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_RemoveRemoteRenderer
00015fb0 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_SetCallback
00015ea8 T Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_SetExternalMediaCodecDecoderRenderer
...

so after changing all my DllImport to using the wrangled names it is now loading the lib and registering all the functions.

I am running with the following mono diagnostics flags

adb shell setprop debug.checkjni 1
adb shell setprop debug.mono.env MONO_LOG_LEVEL=info 
adb shell setprop debug.mono.log gref,gc 

I am now stuck at lines 440-446 in the attached log

2-21 09:58:56.126 I/MonoDroid(18816): UNHANDLED EXCEPTION: System.Runtime.InteropServices.MarshalDirectiveException: Type Java.Lang.Object which is passed to unmanaged code must have a StructLayout attribute.
12-21 09:58:56.126 I/MonoDroid(18816): at (wrapper managed-to-native) WebRtc.ViEAndroidJavaAPI.Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit (Android.Content.Context) 
12-21 09:58:56.126 I/MonoDroid(18816): at WebRtc.ViEAndroidJavaAPI..ctor (Android.Content.Context) [0x00033] in d:\Downloads\WEBRTC\WebRtc\ViEAndroidJavaAPI.cs:30
12-21 09:58:56.126 I/MonoDroid(18816): at WebRtc.WebRTCDemo.startMain () [0x0004b] in d:\Downloads\WEBRTC\WebRtc\WebRTCDemo.cs:536
12-21 09:58:56.126 I/MonoDroid(18816): at WebRtc.WebRTCDemo.OnCreate (Android.OS.Bundle) [0x0028e] in d:\Downloads\WEBRTC\WebRtc\WebRTCDemo.cs:316
12-21 09:58:56.126 I/MonoDroid(18816): at Android.App.Activity.n_OnCreate_Landroid_os_Bundle_ (intptr,intptr,intptr) [0x00011] in /Users/builder/data/lanes/monodroid-mlion-monodroid-4.10.1-branch/d23a19bf/source/monodroid/src/Mono.Android/platforms/android-17/src/generated/Android.App.Activity.cs:2119
12-21 09:58:56.126 I/MonoDroid(18816): at (wrapper dynamic-method) object.468078ad-6f7a-4c7d-8532-4ad45f0259ae (intptr,intptr,intptr) 


After some Googling I reached one of Jonathan Pryor's postings http://forums.xamarin.com/discussion/325/native-library-integration

So my question is:

I was hoping to no have to use any Java Bindings Libraries but instead port the code to C# fully.

If I understand Jonathan's posting correctly, I cannot do it this way that I am doing, but instead I think I have 2 options

Making a Java Bindings Library that contains the needed jars (some of the files that are needed are not in jar files in the Java demo code, so I will have to modify the upstream WebRTC build process to make this work (which was something I hoped to avoid).

Do I have any other options, like manually registering the native methods using JNI?

Regards and thank you in advance
Kenneth Thorman


*** File excerpts supporting posting ***



webrtc\video_engine\test\android\src\org\webrtc\videoengineapp\ViEAndroidJavaAPI.java

public class ViEAndroidJavaAPI {

    public ViEAndroidJavaAPI(Context context) {
        Log.d("*WEBRTCJ*", "Loading ViEAndroidJavaAPI...");
        System.loadLibrary("webrtc-video-demo-jni");

        Log.d("*WEBRTCJ*", "Calling native init...");
        if (!NativeInit(context)) {
            Log.e("*WEBRTCJ*", "Native init failed");
            throw new RuntimeException("Native init failed");
        }
        else {
            Log.d("*WEBRTCJ*", "Native init successful");
        }
        String a = "";
        a.getBytes();
    }

    // API Native
    private native boolean NativeInit(Context context);

    // Video Engine API
    // Initialization and Termination functions
    public native int GetVideoEngine();
    public native int Init(boolean enableTrace);
    public native int Terminate();
...

C# port

    public class ViEAndroidJavaAPI
    {

        public ViEAndroidJavaAPI(Context context)
        {
            Log.Debug("*WEBRTCJ*", "Loading ViEAndroidJavaAPI...");
            Log.Debug("*WEBRTCJ*", "Calling native init...");
            JavaSystem.LoadLibrary("webrtc-video-demo-jni");

            if (!Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit(context))
            {
                Log.Error("*WEBRTCJ*", "Native init failed");
                throw new Exception("Native init failed");
            }
            else
            {
                Log.Debug("*WEBRTCJ*", "Native init successful");
            }
            string a = "";
            Encoding.Default.GetBytes(a);
        }

        // API Native

        [DllImport("libwebrtc-video-demo-jni.so")]
        private static extern bool Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_NativeInit(Context context);

        // Video Engine API
        // Initialization and Termination functions
        [DllImport("libwebrtc-video-demo-jni.so")]
        public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_GetVideoEngine();

        [DllImport("libwebrtc-video-demo-jni.so")]
        public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_Init(bool enableTrace);

        [DllImport("libwebrtc-video-demo-jni.so")]
        public static extern int Java_org_webrtc_videoengineapp_ViEAndroidJavaAPI_Terminate();
...

Original Java sources at

Main app code
I cannot find this in the code.google.com repo so attaching the versions that I have of the Java originating files videoengineapp.zip
 

Supporting libs
https://code.google.com/p/webrtc/source/browse/#svn/trunk/webrtc/modules/audio_device/android/java/src/org/webrtc/voiceengine
https://code.google.com/p/webrtc/source/browse/#svn/trunk/webrtc/modules/video_capture/android/java/src/org/webrtc/videoengine
https://code.google.com/p/webrtc/source/browse/#svn/trunk/webrtc/modules/video_render/android/java/src/org/webrtc/videoengine


Regards and thank you in advance (2nd time - I know :) )
Kenneth Thorman