WarningException on System.DateTime when using .NET Native Compilation

[RCTycooner]

Working with Ceras 4.1.7 in an UWP project (targetting build 16299 of Win10).

With .NET Native tool chain compilation disabled, everything works fine.
However, with that option enabled (and I need it for release builds), I'm getting the following WarningException from Ceras:

Ceras.Exceptions.WarningException: Warning: The type 'System.DateTime' is a struct and by default Ceras serializes structs only through their fields. This struct has only auto-properties, for which the compiler generates hidden 'backing-fields' that are all marked as 'CompilerGenerated'.
You can do multiple things to fix this:
(1) Change all properties to fields.
(2) Explicitly include the properties either with the [Include] attribute, or [MemberConfig].
(3) Disable this warning-exception in 'config.Warnings'.
at Ceras.TypeConfig.Seal() + 0x2b8
at Ceras.CerasSerializer.CreatePrimarySchema(Type, Boolean) + 0x90
at Ceras.CerasSerializer.CreateMetaData(Type, Boolean) + 0x184
at Ceras.CerasSerializer.GetReferenceFormatter(Type) + 0x27
at Ceras.Formatters.DynamicFormatter1.GenerateSerializer(CerasSerializer, Schema, Boolean, Boolean) + 0x491 at Ceras.Formatters.DynamicFormatter1.Initialize() + 0x4e
at Ceras.CerasSerializer.PrepareFormatter(IFormatter) + 0x28
at Ceras.CerasSerializer.GetSpecificFormatter(Type, TypeMetaData) + 0x15a
at Ceras.CerasSerializer.GetSpecificFormatter(Type) + 0x2d
at Ceras.Formatters.ReferenceFormatter1.GetOrCreateEntry(Type) + 0xc7 at Ceras.Formatters.ReferenceFormatter1.Deserialize(Byte[], Int32&, T&) + 0x2b7
at Ceras.Formatters.CollectionFormatter2.Deserialize(Byte[], Int32&, TCollection&) + 0x24a at System.Action3.Invoke(T1, T2, T3) + 0x24
at Ceras.Formatters.ReferenceFormatter1.Deserialize(Byte[], Int32&, T&) + 0x372 at _$ILCT$.$ILT$ReflectionDynamicInvoke$.InvokeRetVIRR[T0, T1, T2](Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x74 at System.InvokeUtils.CalliIntrinsics.Call(IntPtr, IntPtr, Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x2f at System.InvokeUtils.CallDynamicInvokeMethod(Object, IntPtr, Object, IntPtr, IntPtr, Object, Object[], BinderBundle, Boolean, Boolean, Boolean) + 0x10c --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c at System.Linq.Expressions.Interpreter.ExceptionHelpers.UnwrapAndRethrow(TargetInvocationException) + 0x14 at System.Linq.Expressions.Interpreter.ByRefMethodInfoCallInstruction.Run(InterpretedFrame) + 0x203 at System.Linq.Expressions.Interpreter.Interpreter.Run(InterpretedFrame) + 0x25 at System.Linq.Expressions.Interpreter.LightLambda.RunVoid(Object[]) + 0x96 at Ceras.Formatters.DeserializeDelegate1.InvokeObjectArrayThunk(Byte[], Int32&, T&) + 0x86
at Ceras.Formatters.DynamicFormatter1.Deserialize(Byte[], Int32&, T&) + 0x24 at System.Action3.Invoke(T1, T2, T3) + 0x24
at Ceras.Formatters.ReferenceFormatter1.Deserialize(Byte[], Int32&, T&) + 0x372 at _$ILCT$.$ILT$ReflectionDynamicInvoke$.InvokeRetVIRR[T0, T1, T2](Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x74 at System.InvokeUtils.CalliIntrinsics.Call(IntPtr, IntPtr, Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x2f at System.InvokeUtils.CallDynamicInvokeMethod(Object, IntPtr, Object, IntPtr, IntPtr, Object, Object[], BinderBundle, Boolean, Boolean, Boolean) + 0x10c --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c at System.Linq.Expressions.Interpreter.ExceptionHelpers.UnwrapAndRethrow(TargetInvocationException) + 0x14 at System.Linq.Expressions.Interpreter.ByRefMethodInfoCallInstruction.Run(InterpretedFrame) + 0x203 at System.Linq.Expressions.Interpreter.Interpreter.Run(InterpretedFrame) + 0x25 at System.Linq.Expressions.Interpreter.LightLambda.RunVoid(Object[]) + 0x96 at Ceras.Formatters.DeserializeDelegate1.InvokeObjectArrayThunk(Byte[], Int32&, T&) + 0x86
at Ceras.Formatters.DynamicFormatter1.Deserialize(Byte[], Int32&, T&) + 0x24 at System.Action3.Invoke(T1, T2, T3) + 0x24
at Ceras.Formatters.ReferenceFormatter1.Deserialize(Byte[], Int32&, T&) + 0x372 at Ceras.Formatters.CollectionFormatter2.Deserialize(Byte[], Int32&, TCollection&) + 0x24a
at System.Action3.Invoke(T1, T2, T3) + 0x24 at Ceras.Formatters.ReferenceFormatter1.Deserialize(Byte[], Int32&, T&) + 0x372
at _$ILCT$.$ILT$ReflectionDynamicInvoke$.InvokeRetVIRR[T0, T1, T2](Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x74
at System.InvokeUtils.CalliIntrinsics.Call(IntPtr, IntPtr, Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x2f
at System.InvokeUtils.CallDynamicInvokeMethod(Object, IntPtr, Object, IntPtr, IntPtr, Object, Object[], BinderBundle, Boolean, Boolean, Boolean) + 0x10c
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
at System.Linq.Expressions.Interpreter.ExceptionHelpers.UnwrapAndRethrow(TargetInvocationException) + 0x14
at System.Linq.Expressions.Interpreter.ByRefMethodInfoCallInstruction.Run(InterpretedFrame) + 0x203
at System.Linq.Expressions.Interpreter.Interpreter.Run(InterpretedFrame) + 0x25
at System.Linq.Expressions.Interpreter.LightLambda.RunVoid(Object[]) + 0x96
at Ceras.Formatters.DeserializeDelegate1.InvokeObjectArrayThunk(Byte[], Int32&, T&) + 0x86 at Ceras.Formatters.DynamicFormatter1.Deserialize(Byte[], Int32&, T&) + 0x24
at System.Action3.Invoke(T1, T2, T3) + 0x24 at Ceras.Formatters.ReferenceFormatter1.Deserialize(Byte[], Int32&, T&) + 0x372
at _$ILCT$.$ILT$ReflectionDynamicInvoke$.InvokeRetVIRR[T0, T1, T2](Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x74
at System.InvokeUtils.CalliIntrinsics.Call(IntPtr, IntPtr, Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x2f
at System.InvokeUtils.CallDynamicInvokeMethod(Object, IntPtr, Object, IntPtr, IntPtr, Object, Object[], BinderBundle, Boolean, Boolean, Boolean) + 0x10c
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
at System.Linq.Expressions.Interpreter.ExceptionHelpers.UnwrapAndRethrow(TargetInvocationException) + 0x14
at System.Linq.Expressions.Interpreter.ByRefMethodInfoCallInstruction.Run(InterpretedFrame) + 0x203
at System.Linq.Expressions.Interpreter.Interpreter.Run(InterpretedFrame) + 0x25
at System.Linq.Expressions.Interpreter.LightLambda.RunVoid(Object[]) + 0x96
at Ceras.Formatters.DeserializeDelegate1.InvokeObjectArrayThunk(Byte[], Int32&, T&) + 0x86 at Ceras.Formatters.DynamicFormatter1.Deserialize(Byte[], Int32&, T&) + 0x24
at System.Action3.Invoke(T1, T2, T3) + 0x24 at Ceras.Formatters.ReferenceFormatter1.Deserialize(Byte[], Int32&, T&) + 0x372
at _$ILCT$.$ILT$ReflectionDynamicInvoke$.InvokeRetVIRR[T0, T1, T2](Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x74
at System.InvokeUtils.CalliIntrinsics.Call(IntPtr, IntPtr, Object, IntPtr, InvokeUtils.ArgSetupState&, Boolean) + 0x2f
at System.InvokeUtils.CallDynamicInvokeMethod(Object, IntPtr, Object, IntPtr, IntPtr, Object, Object[], BinderBundle, Boolean, Boolean, Boolean) + 0x10c
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x1c
at System.Linq.Expressions.Interpreter.ExceptionHelpers.UnwrapAndRethrow(TargetInvocationException) + 0x14
at System.Linq.Expressions.Interpreter.ByRefMethodInfoCallInstruction.Run(InterpretedFrame) + 0x203
at System.Linq.Expressions.Interpreter.Interpreter.Run(InterpretedFrame) + 0x25
at System.Linq.Expressions.Interpreter.LightLambda.RunVoid(Object[]) + 0x96
at Ceras.Formatters.DeserializeDelegate1.InvokeObjectArrayThunk(Byte[], Int32&, T&) + 0x86 at Ceras.Formatters.DynamicFormatter1.Deserialize(Byte[], Int32&, T&) + 0x24
at System.Action3.Invoke(T1, T2, T3) + 0x24 at Ceras.Formatters.ReferenceFormatter1.Deserialize(Byte[], Int32&, T&) + 0x372
at Ceras.CerasSerializer.Deserialize[T](T&, Byte[], Int32&, Int32) + 0x131
at Ceras.CerasSerializer.DeserializeT + 0x35
at Calidos.Maat.ClientData.ServiceClient.WebApiCerasBroker.DeSerializeT + 0x5d

All my classes are marked with [MemberConfig(TargetMember.AllProperties)]

How can I resolve this?
Thx,
Tom

[RCTycooner]

I was able to work around this issue by marking the DateTime-properies with Exclude, and adding a string-property that wraps the original DateTime prop.

Something like:

[Include()]
public String ValidFromAsString
{
get
{
return this.ValidFrom.ToString("yyyy-MM-dd");
}
set
{
this.ValidFrom = DateTime.ParseExact(value, "yyyy-MM-dd", CultureInfo.InvariantCulture);
}
}

[rikimaru0345]

DateTime can't be serialized when using .NET Native?
I have little experience with .NET Native, so I don't know what it is doing to DateTime behind the scenes.

We could bypass the problem by creating a custom formatter for this type, but the same issue will likely pop up with other types.

For now, you can try just including the two relevant props:

• DateTimeKind Kind
• long Ticks

Like this:

config.ConfigType<DateTime>()
    .ConfigMember(p => p.Kind).Include()
    .ConfigMember(p => p.Ticks).Include();

Definitely let me know if that helps, and if you get a similar problem with other types.
Maybe we can identify a common pattern and automatically apply this sort of inclusion.

[RCTycooner]

I can confirm this works when marking the Kind & Ticks properties with Include as you've suggested.

For now, this has been the only type I've seen this error on.

(We were using JSON (and XML) serialization before Ceras, never had any issues with DateTimes.)

This issue isn't a priority for us anymore, as I'm using the "AsString" variant I described above. This allows me to solve another issue regarding differences in timezones when passing dates between my server and client.

[rikimaru0345]

I can confirm this works when marking the Kind & Ticks properties with Include as you've suggested.

Thanks for testing that! 😄 ❤️
Could you also try adding the following setting (together with that .Include() workaround)

config.ConfigType<DateTime>()
      .CustomResolver = (c, t) => c.Advanced
          .GetFormatterResolver<DynamicObjectFormatterResolver>()
          .GetFormatter(t);

Does Ceras still round-trip a DateTime value correctly when running that in .net native?
No exceptions? No corrupted or lost data?

We were using JSON (and XML) serialization before Ceras, never had any issues with DateTimes.

So I've looked into all of this some more...

Since the definition for DateTime consists of a single field https://github.com/microsoft/referencesource/blob/17b97365645da62cf8a49444d979f94a59bbb155/mscorlib/system/datetime.cs#L131-L140
that warning usually doesn't get triggered.

But it seems the UWP libraries are written in a slightly different syntax, so that there's no explicit/manual field definition! So when Ceras "discoveres" the type and creates the TypeConfig for it, that check gets triggered.

Like other serializers, Ceras also uses a Formatter made specifically for DateTime:

Ceras/src/Ceras/Resolvers/StandardFormatterResolver.cs

Lines 199 to 212 in cc24d60

	class DateTimeFormatter : IFormatter<DateTime>
	{
	public void Serialize(ref byte[] buffer, ref int offset, DateTime value)
	{
	var v = value.ToBinary();
	WriteInt64Fixed(ref buffer, ref offset, v);
	}
	public void Deserialize(byte[] buffer, ref int offset, ref DateTime value)
	{
	var v = ReadInt64Fixed(buffer, ref offset);
	value = DateTime.FromBinary(v);
	}
	}

I'm hesitant about going with the straight-forward approach (just disabling the check for types that Ceras can handle with a built-in formatter), because the user can always override the formatter.

But then again maybe that's a rare / advanced scenario and someone who changes the formatter for a built-in type is probably aware of the potential issues.

Maybe the easiest solution would be to just make an exception for DateTime, treating it as a special case? :/

[RCTycooner]

Hmm,

To test it this time, I've written a small program. But now the previous workaround (where you mark the Kind and Ticks properties to be included) no longer works!
Ceras.Serialize() is throwing the following exception there:

property must be readable and writable

It is referring to the property "Kind" here. But I suspect it will also throw this error on the property Ticks. I saw in the DateTime class that both only have a getter, and no setter.
No idea why this was working earlier today! (I didn't use a test-project, and just putted the config in our main solution)

If I try your new config, it throws the original WarningException:
Warning: The type 'System.DateTime' is a struct and by default Ceras serializes structs only through their fields. This struct has only auto-properties, for which the compiler generates hidden 'backing-fields' that are all marked as 'CompilerGenerated'. You can do multiple things to fix this: (1) Change all properties to fields. (2) Explicitly include the properties either with the [Include] attribute, or [MemberConfig]. (3) Disable this warning-exception in 'config.Warnings'.

Trying a config that tries to include the field dateData fails as ConfigField() returns null.

I've published my sample app here: https://github.com/RCTycooner/CerasTryouts
If you run it in Debug/x86, it will use .NET Native compilation. The various scenario's we've tried can be found in the file https://github.com/RCTycooner/CerasTryouts/blob/master/CerasTryouts/CerasTryouts/CerasTryer.cs

Let me know if I can help with anything regarding this. As stated before, our production code will be using the wrapped "AsString" property.

[rikimaru0345]

But now the previous workaround (where you mark the Kind and Ticks properties to be included) no longer works!

The resolver part was missing from there.

When you set those two fields to be included that only changed the TypeConfig.
That's why the warning didn't appear again (it is triggered from inside the TypeConfig while it validates itself).

That means it was still using the built-in DateTimeFormatter, which doesn't respect the TypeConfig as you can see from its code (its extremely simple).

Most settings in TypeConfig are only relevant to DynamicObjectFormatter<>. (well, user formatters could use those settings as well in theory)

Ceras.Serialize() is throwing the following exception there:

I see, seems like those two members really are "getter-only properties".

Trying a config that tries to include the field dateData fails as ConfigField() returns null.

In that case a field with that name doesn't exist.
I looked it up:

• dateData for net framework
• _dateData for .net core
• and in .net native it seems to be written as a getter only property.

In any case, the built-in formatter that is used by default should work perfectly on any platform.

Let me know if I can help with anything regarding this.

The only thing left to do is deciding on how to handle this in the future.
Should I word the warning-exception differently?
Or add that little if() special case / exception for DateTime that I mentioned before?
What do you think?

[RCTycooner]

The ideal situation is ofcourse that basic types such as a DateTime work out of the box. So I would add a special condition for DateTimes.