NUnit tests works from commandline (nunit-console) but not from Xamarin Studio

I've tried to set up a NUnit Library Project in our solution. In the project I have a test class with one test case/method and all it do is create a new instance of my view model from another project, take out a list property and assert the length of the list and the contents of a title property of the first object in the list. So nothing fancy is going on, but when I run all unit tests in Xamarin Studio it marks the test case as failed and the only error message I get is the following:

System.IO.FileNotFoundException : Could not load file or assembly 'System, Version=2.0.5.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e' or one of its dependencies.
  at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00054] in /private/tmp/source/bockbuild-xamarin/profiles/mono-mac-xamarin-no-pcl/build-root/mono-3.2.2/mcs/class/corlib/System.Reflection/MonoMethod.cs:230

When I run the test from command line with nunit-console and point it at the .dll that Xamarin Studio built it succeeds.

nunit-console NiceProject.Test/bin/Debug/NiceProject.Test.dll -xml:nunit-report.xml

I've tried the above with both nunit.framework from 'Edit References' in the project and the one found by the NuGet plugin.

I'm using the newest Xamarin Studio from alpha channel.