ProtoException when deserializing class with member marked with AsReference removed

Question

I get a ProtoException

ProtoBuf.ProtoException : Internal error; a key mismatch occurred

with the following code:

[ProtoContract]
class Foo { }

class MemberRemovedTest
{
    [ProtoContract]
    class V1
    {
        [ProtoMember(1, AsReference = true)]
        public Foo A { get; set; }

        [ProtoMember(2, AsReference =  true)]
        public Foo B { get; set; }
    }

    [ProtoContract]
    class V2
    {
        [ProtoMember(2, AsReference = true)]
        public Foo B { get; set; }
    }

    public void BasicTest()
    {
        var v1 = new V1();
        v1.A = new Foo();
        v1.B = new Foo();

        byte[] buffer;
        V2 v2;
        using (var stream = new MemoryStream())
        {
            Serializer.Serialize(stream, A);
            buffer = stream.ToArray();
        }
        using (var stream = new MemoryStream(buffer))
        {
            v2 = Serializer.Deserialize<V2>(stream); //Exception here
        }
    }
}

It will not throw an exception if:

  • The second member is the one removed.
  • The ProtoMember attributes of A or B are not AsReference = true.
  • A and B are not both set with a Foo instance.

I am of the understanding that protobuf supports member removal, but this seems to indicate there are cases where they must be kept around.

Is this a bug in Protobuf or a bad assumption about removing members?

Call stack for exception:

   at ProtoBuf.NetObjectCache.SetKeyedObject(Int32 key, Object value) in c:\Dev\protobuf-net\protobuf-net\NetObjectCache.cs: line 67
   at ProtoBuf.BclHelpers.ReadNetObject(Object value, ProtoReader source, Int32 key, Type type, NetObjectOptions options) in c:\Dev\protobuf-net\protobuf-net\BclHelpers.cs: line 425
   at proto_6(Object, ProtoReader)
   at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source) in c:\Dev\protobuf-net\protobuf-net\Serializers\CompiledSerializer.cs: line 57
   at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source) in c:\Dev\protobuf-net\protobuf-net\Meta\RuntimeTypeModel.cs: line 715
   at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type type, Object value, Boolean noAutoCreate) in c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs: line 679
   at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type, SerializationContext context) in c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs: line 580
   at ProtoBuf.Serializer.Deserialize(Stream source) in c:\Dev\protobuf-net\protobuf-net\Serializer.cs: line 77
   at ####.ProtoBuf.MemberRemovedTest.BasicTest() in MemberRemovedTest.cs: line 56
Solution

Hmmm.... Yes, interesting. Damn. Making that scenario work could be extremely problematic; without the member, we don't have enough metadata to deserialize the object - or even just to know that it is an object that might be serving as a place-holder for an as-reference object that occurs later. It would be impractical to store every skipped field for processing later on - and indeed, in non-trivial cases it would be impossible for technical reasons (if the as-reference is a sub-object, potentially several levels down, of an object that is removed as a member higher up - we can't process that due to missing metadata).

I do not have a better comment right now than "yes, that won't work". Vexing - interesting scenario.