Deserialize object serialized with older assembly version

[MichalJakubeczy]

We are serializing objects of assembly with version e.g. 1.0.125.10594, these are also stored to Redis cache. Then we deploy a new release (with new version of DLLs, e.g. 1.0.126.11000) and we observe FileLoadException.

Stacktrace:

System.IO.FileLoadException: Could not load file or assembly '{xxx}.Common, Version=1.0.125.10594, Culture=neutral, PublicKeyToken=8de2f378b16e1f44' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040) 
File name: '{xxx}.Common, Version=1.0.125.10594, Culture=neutral, PublicKeyToken=8de2f378b16e1f44' 
   at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMarkHandle stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName, ObjectHandleOnStack type) 
   at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName) 
   at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, StackCrawlMark& stackMark, Boolean loadTypeFromPartialName) 
   at System.Type.GetType(String typeName, Boolean throwOnError) 
   at CacheManager.Redis.RedisValueConverter.GetType(String type) 
   at CacheManager.Redis.RedisValueConverter.CacheManager.Redis.IRedisValueConverter<System.Object>.FromRedisValue(RedisValue value, String type) 
   at CacheManager.Redis.RedisCacheHandle`1.FromRedisValue(RedisValue value, String valueType) 
   at CacheManager.Redis.RedisCacheHandle`1.<>c__DisplayClass35_0.<GetCacheItemInternal>b__0() 
   at CacheManager.Redis.RetryHelper.Retry[T](Func`1 retryme, Int32 timeOut, Int32 retries, ILogger logger) 
   at CacheManager.Redis.RedisCacheHandle`1.Retry[T](Func`1 retryme) 
   at CacheManager.Core.BaseCacheManager`1.GetCacheItemInternal(String key, String region) 
   at CacheManager.Core.Internal.BaseCache`1.Get(String key) 
   at CacheManager.Core.Internal.BaseCache`1.Get[TOut](String key) 
   at {xxx}.Infrastructure.Common.Caching.AzureCacheStorage.GetObject[TObject](String key, Func`1 loader) 

There seems to be a problem that during deserialization of value from cache old assembly is not found (obviously) and object can not be deserialized and retrieved.

We configure CacheManager like this:

                        // inspired by http:https://cachemanager.net/
                        // section "Using In-Memory and Redis"
                        settings
                            .WithSystemRuntimeCacheHandle()
                            .WithExpiration(CacheManager.Core.ExpirationMode.Absolute, defaultExpiryTime)
                            .And
                            .WithRedisConfiguration(CacheManagerRedisConfigurationKey, connectionString)
                            .WithMaxRetries(3)
                            .WithRetryTimeout(100)
                            .WithJsonSerializer()
                            .WithRedisBackplane(CacheManagerRedisConfigurationKey)
                            .WithRedisCacheHandle(CacheManagerRedisConfigurationKey, true)
                            .WithExpiration(CacheManager.Core.ExpirationMode.Absolute, defaultExpiryTime);

so according to documentation JSON should be version agnostic.

Where might the problem lay?

[MichaCo]

Hi @MichalJakubeczy yes, I have to store the actual type of the object stored in Redis for example, because, otherwise, the de/serialization will be very difficult, especially if the generic type of CacheManager doesn't match the type stored.

I just did some testing around this and found that unsigned assemblies actually work just fine.
Only if you use signed assemblies, the versions have to match and Type.GetType actually throws an exception...

For now, you can try to use unsigned assemblies to work around that issue.

I will have to implement custom type resolvers and maybe even expose an interface or something so that you could inject your custom type resolver to work around those things...
But this will take some time to get it into the library...

Hope that helps! Sorry that I cannot give you a better fix right away.

[MichalJakubeczy]

Thanks @MichaCo for your effort. Unfortunately we cannot use unsigned assemblies right away.

One more workaround came to my mind - adding assembly version to the cache key would separate old values from the new ones. Old ones will disapear as they don't get accessed (key with old assembly version will not get accessed) and new ones will be added.

It will cause some performance drop right after the release of new version (we can't use cached stuff), but better than nothing.

[MichaCo]

@MichalJakubeczy I just added a few adjustments to the way I'm loading the types.
First of all, I added more fallbacks and the implementation now also tries to strip out the version number from the type string and tries to load that. This at least worked for me even for signed assemblies!

If something still doesn't work for you, there is a new method to add custom type resolvers.
Example code:

CacheManager.Core.Internal.TypeCache.RegisterResolveType((name) =>
{
    return null;
});

The method is allowed to return null or throw an exception without breaking the logic.

It would be great if you could try to consume the latest beta package from the myget feed (0.9.4-beta-1446 has the latest bits) and test that with your case and let me know if the standard resolver (without custom resolver) now works.

Thanks!
m

[MichalJakubeczy]

@MichaCo - wow, thanks for a quick response. I will give it a try in upcoming days and let you know about the result.

Is week fast enough?

[MichaCo]

@MichalJakubeczy sure np ;)

[MichalJakubeczy]

@MichaCo - I tested it and it seems to be working :) Could you please make a release via nuget, so we can start using it in our solution.

Thanks.

[MichaCo]

@MichalJakubeczy I decided to release a beta of all packages to Nuget. You can consume those and see how it goes.

I will need a little bit more time/testing to wrap that 1.0.0 release up. Mostly because of the move to VS2017 project system.
Happy testing!

[MichalJakubeczy]

@MichaCo OK, thanks for letting me know. I will carry on using beta in mine local environment and let you know about any issues. But we can't use beta stuff in Production environment for obvious reasons :)

Will 1.0.0 contain this fix as well (if nothing unexpected happens)?

[MichaCo]

yeah 1.0.0 does have all the fixes/changes. Unfortunately there are some issues with the packages. Dependencies don't get resolved correctly.

Ok if you cannot use beta stuff anyways then you have to wait a little bit ;) Hopefully before/through the weekend I'll have time to get this release done

[zorrme]

Where do we specify this in a DI environment ?

CacheManager.Core.Internal.TypeCache.RegisterResolveType((name) =>
{
return null;
});

[MichaCo]

TypeCache is just a static thing, not a dependency. nothing to inject...

Why do you think you have to use this function? It should be just a "workaround" if the build in TypeLoader doesn't work in some serialization scenarios. So, if you have a particular issue with something, would be interesting to know.

[zorrme]

I am using cachemanager in web api project and .net core with json serializer. I do not want to store type in redis. Shouldn't json should be type agnostic ?

[zorrme]

I have defined:

CacheManager.Core.Internal.TypeCache.RegisterResolveType((name) =>
{
return null;
});

But the type is still showing up redis.

[MichaCo]

@zorrme I think you misunderstand the purpose of the TypeCache.
It is there to resolve types by name and not to modify what CacheManager stores inside the cache (Redis or whatever).

CacheManager needs to store the type's name of the actual cached value separated to the value, so that when it read the data the next time, it can deserialize it into the correct type. The generic CacheValue<T> could have a slightly different T then the value actually stored, object for example...

In short, what you are trying to do is not supported.
Write your own serializer or simply store strings only instead...

[zorrme]

@MichaCo Thanks for the suggestion, but unfortunately string is also stored as a type. Can we have a version of cache manager which does not store types ?
Also, I am not quite sure about the need to store type. Are we not able to use type for deserializing ?

[MichaCo]

@zorrme Just curious, but what is the problem with storing the type name as one of the meta data in Redis? I honestly don't get it.

Can we have a version of cache manager which does not store types ?

No.

Also, I am not quite sure about the need to store type. Are we not able to use type for deserializing?

I explained that in the last answer, there is no way to guess the type when reading a json object.
The solution here is to store the type which was used to serialize the value.

[zorrme]

@MichaCo I am using two apps to store same POCO in redis.
One app is written in web api (.net 4.6) and the other one is in .net core.

If we write to redis from app1, app2 cannot read it because of type mismatch and vice-versa.

[MichaCo]

@zorrme It it would really be the same POCO, it would probably work (shared library even with different versions now).

Just guessing here, but, you are defining the POCO twice, in each APP? Meaning the type is actually totally different?
That's simply not be supported by the CacheManager serialization, and will never be.

Solutions:
De/serialize it yourself and cache it as string or byte[]. (that's what I meant with cache as string).
Or, try to use TypeCache.RegisterResolveType to return the correct typeof(POCO) each time if it asks for same name but different assembly. That might or might not work (never tested or support that).

If you still have issues, please use StackOverflow with example code to reproduce it.