System.InvalidOperationException: Cannot read Int64 as Bool

[vismein]

Hi,

I am using latest version of MarcelloDB - 1.0.8 and getting an exception (intermittently) on following scenarios -

1. FindAll()
   public IEnumerable FindAll(Func<T, bool> predicate)
   {
   lock (_lockSessionAccess)
   {
   return GetCollection().All.Where(predicate);
   }
   }
2. Add()
   public virtual void Add(T entity, bool autoflush = true)
   {
   lock (_lockSessionAccess)
   {
   GetCollection().Persist(entity);
   if (autoflush) Flush();
   }
   }

Exception Stack Trace -

Exception:System.InvalidOperationException: Cannot read Int64 as Bool
at MarcelloDB.Serialization.BinaryFormatter.ReadValidTypeID (MarcelloDB.Serialization.BinaryFormatter+TypeID wantedTypeID) [0x0003b] in :0
at MarcelloDB.Serialization.BinaryFormatter.ReadBool () [0x00001] in :0
at MarcelloDB.Serialization.ValueSerializers.BooleanSerializer.ReadValue (MarcelloDB.Serialization.BinaryFormatter formatter) [0x00001] in :0
at MarcelloDB.Serialization.BTreeNodeBinaryFormatterSerializer1[TKey].ReadEntry (MarcelloDB.Serialization.BinaryFormatter formatter) [0x0001b] in <ce43c20e305646cf958407e87464f571>:0 at MarcelloDB.Serialization.BTreeNodeBinaryFormatterSerializer1[TKey].ReadEntries (MarcelloDB.Serialization.BinaryFormatter formatter) [0x00013] in :0
at MarcelloDB.Serialization.BTreeNodeBinaryFormatterSerializer1[TKey].Deserialize (System.Byte[] bytes) [0x00023] in <ce43c20e305646cf958407e87464f571>:0 at MarcelloDB.Index.BTree.RecordBTreeDataProvider1[TNodeKey].GetNode (System.Int64 address) [0x0003a] in :0
at MarcelloDB.Index.BTree.RecordBTreeDataProvider1[TNodeKey].GetRootNode (System.Int32 degree) [0x00040] in <ce43c20e305646cf958407e87464f571>:0 at MarcelloDB.Index.BTree.BTreeWalker1[TK].Reset () [0x00022] in :0
at MarcelloDB.Index.BTree.BTreeWalker1[TK]..ctor (System.Int32 degree, MarcelloDB.Index.BTree.IBTreeDataProvider1[TK] dataProvider) [0x00024] in :0
at MarcelloDB.Index.RecordIndex1[TNodeKey].GetWalker () [0x00008] in <ce43c20e305646cf958407e87464f571>:0 at MarcelloDB.Collections.BaseIndexedValue2[TObj,TAttribute].EnsureIndex () [0x00045] in :0
at MarcelloDB.Collections.Collection2[T,TID].Initialize () [0x0001e] in <ce43c20e305646cf958407e87464f571>:0 at MarcelloDB.Collections.CollectionFile.Collection[T,TID,TIndexDef] (System.String collectionName, System.Func2[T,TResult] idFunc) [0x0005a] in :0
at TXX.Xamarin.Forms.Persistence.NoSQL.Abstract.IndexedRepositoryBase3[T,TKey,TIndex].GetCollection () [0x00012] in <ba352e7d3b8c4296ac040d6fc327dc5c>:0 at TXX.Xamarin.Forms.Persistence.NoSQL.Abstract.IndexedRepositoryBase3[T,TKey,TIndex].FindByKey (TKey key) [0x00012] in :0
at Mobile.Infrastructure.Persistence.NoSQL.TasksRepository.Add (Mobile.Entity.Tasks.Abstract.TaskDefinition entity, System.Boolean autoflush) [0x00019] in :0
at Mobile.Infrastructure.BackgroundTasks.TaskManager.RegisterStoreAndForward (Mobile.Entity.Tasks.Abstract.TaskDefinition task) [0x000d0] in :0 {"TaskType":18,"ProNumber":"","TaskId":"51669153-3476-4932-85b2-0aaa9efc229b","RegisteredDate":"2019-12-18T13:28:57.744949-05:00","Completed":false,"ExecutionDate":"2019-12-18T13:28:57.745142-05:00","StatusCode":0,"ExternalMoveId":null,"CurrentEvent":0,"User":"10302t","Password":"","Latitude":10.816047,"Longitude":79.464586,"Accuracy":"5.098404","LocationProvider":"gps","FormattedDate":"20191218 13:28:47","Timezone":"EST","ActivityName":"GEOLOCATION","ExecutionTimes":1,"Message":null,"ResultCode":0,"ActivityCode":"SDGL","Details":null,"Version":""}

I am using the IndexedValue as below

public class TaskIndex : IndexDefinition
{
public IndexedValue<TaskDefinition, bool> Completed { get; set; }
}

Please help. Any insight to this issue is appreciated.

Thanks

[markmeeus]

Thank you for reporting this.
I'm a bit puzzled at what is happening here.

First of all, the index definition you posted doesn't seem correct, IndexDefinition is a generic type and can't really be derived without a type arg. Also, even if you would be able to create such an index, the collection method only allows index definitions for the same type as the one used for the collection.

So let's say you have

class Task {
  public Completed { get; set; }
}

and you want the collection to create an index on ```Competed````
you would need:

class TaskIndex: IndexDefinition<Task> {
  public  IndexedValue<Task, bool> Completed { get; set; }
}

I did a basic test to see if there was something wrong with indexing bool fields, but that seems ok.

Could you give me some more info about when it occurs.
For instance, is it reproducable?
How many items do you have in the collection when it happens?

As as side note, using indexes gives you a benefit for searching items in the list. However, if you iterate the .All directly on the collection, you won't have this benefit because it will iterate all items anyway.

[vismein]

Hello Mark,

Sorry. My bad. The actual index definition looks below -
public class TaskIndex : IndexDefinition<TaskDefinition> { public IndexedValue<TaskDefinition, bool> Completed { get; set; } }
Answers to your questions -

Could you give me some more info about when it occurs.
For instance, is it reproducable? - It is intermittent in nature, but Yes. I was able to get the same exception today as well while debugging. It usually happens when my application works offline and we are storing the tasks carried out during that network outage. We fetch these data and show it in a listview. Once the network reinstates, we mark the completed value as true and remove that entry from collection. The operations where such an exception is seen are

1. Querying using Linq with the index item as part of the predicate.
   eg: -

• `var tasks = _tasksRepository.FindAll(x => x != null && x.ExternalMoveId != null

                                        && x.ExternalMoveId == externalMoveId && !x.Completed).ToList();
  

`

• tasks = _tasksRepository.FindAll(x => !x.Completed).OrderBy(x => x.RegisteredDate);

2. When adding/updating/deleting objects to the same repository
   eg:-
   _tasksRepository.Add(task);

Implementation of FindAll() , Add()/Update() is as below

public IEnumerable<T> FindAll(Func<T, bool> predicate) { lock (_lockSessionAccess) { return GetCollection().All.Where(predicate); } }
`public T FindByKey(TKey key)
{
lock (_lockSessionAccess)
{
return GetCollection().Find(key);
}
}

    public virtual void Add(T entity, bool autoflush = true)
    {
        lock (_lockSessionAccess)
        {
            GetCollection().Persist(entity);
            if (autoflush) Flush();
        }
    }

`
How many items do you have in the collection when it happens? - I have seen issue with just 2 items in the collection.

I have noticed that in all of the stack traces, the FindAll() or FindByKey() are common.

Another Instance of this exception which got reproduced while debugging.
[MonoDroid] System.InvalidOperationException: Cannot read Int64 as Bool
[MonoDroid]
at MarcelloDB.Serialization.BinaryFormatter.ReadValidTypeID (MarcelloDB.Serialization.BinaryFormatter+TypeID wantedTypeID) [0x0003b] in :0
[MonoDroid]
at MarcelloDB.Serialization.BinaryFormatter.ReadBool () [0x00001] in :0
[MonoDroid]
at MarcelloDB.Serialization.ValueSerializers.BooleanSerializer.ReadValue (MarcelloDB.Serialization.BinaryFormatter formatter) [0x00001] in :0
[MonoDroid]
at MarcelloDB.Serialization.BTreeNodeBinaryFormatterSerializer1[TKey].ReadEntry (MarcelloDB.Serialization.BinaryFormatter formatter) [0x0001b] in :0 [MonoDroid] at MarcelloDB.Serialization.BTreeNodeBinaryFormatterSerializer1[TKey].ReadEntries (MarcelloDB.Serialization.BinaryFormatter formatter) [0x00013] in :0
[MonoDroid]
at MarcelloDB.Serialization.BTreeNodeBinaryFormatterSerializer1[TKey].Deserialize (System.Byte[] bytes) [0x00023] in :0 [MonoDroid] at MarcelloDB.Index.BTree.RecordBTreeDataProvider1[TNodeKey].GetNode (System.Int64 address) [0x0003a] in :0
[MonoDroid]
at MarcelloDB.Index.BTree.RecordBTreeDataProvider1[TNodeKey].GetRootNode (System.Int32 degree) [0x00040] in :0 [MonoDroid] at MarcelloDB.Index.BTree.BTreeWalker1[TK].Reset () [0x00022] in :0
[MonoDroid]
at MarcelloDB.Index.BTree.BTreeWalker1[TK]..ctor (System.Int32 degree, MarcelloDB.Index.BTree.IBTreeDataProvider1[TK] dataProvider) [0x00024] in :0
[MonoDroid]
at MarcelloDB.Index.RecordIndex1[TNodeKey].GetWalker () [0x00008] in :0 [MonoDroid] at MarcelloDB.Collections.BaseIndexedValue2[TObj,TAttribute].EnsureIndex () [0x00045] in :0
[MonoDroid]
at MarcelloDB.Collections.Collection2[T,TID].Initialize () [0x0001e] in :0 [MonoDroid] at MarcelloDB.Collections.CollectionFile.Collection[T,TID,TIndexDef] (System.String collectionName, System.Func2[T,TResult] idFunc) [0x0005a] in :0
[MonoDroid]
at TXX..Xamarin.Forms.Persistence.NoSQL.Abstract.IndexedRepositoryBase3[T,TKey,TIndex].GetCollection () [0x00000] in /TXX..Xamarin.Forms.Persistence/NoSQL/Abstract/IndexedRepositoryBase.cs:17 [MonoDroid] at TXX..Xamarin.Forms.Persistence.NoSQL.Abstract.IndexedRepositoryBase3[T,TKey,TIndex].FindAll (System.Func2[T,TResult] predicate) [0x00012] in /TXX..Xamarin.Forms.Persistence/NoSQL/Abstract/IndexedRepositoryBase.cs:50 [MonoDroid] at Mobile.Infrastructure.Persistence.NoSQL.TasksRepository.TXX..Xamarin.Forms.Persistence.NoSQL.Abstract.IRepository.FindAll (System.Func2[T,TResult] predicate) <0x7971dba700 + 0x00043> in <56ed32dfce434bb2b108cfc1ec1a0548>:0
[MonoDroid]
at Mobile.Infrastructure.BackgroundTasks.TaskManager.IsTrippedWorkOrderInQueue (System.String externalMoveId) [0x000c1] in /Users/Source/Mobile.Infrastructure/BackgroundTasks/TaskManager.cs:214
[MonoDroid]
at Mobile.Infrastructure.BackgroundTasks.TaskManager.TryExecuteOnline[TR] (Mobile.Entity.Tasks.Abstract.TaskDefinition task) [0x00012] in /Users/Source/Mobile.Infrastructure/BackgroundTasks/TaskManager.cs:177
[MonoDroid]
at Mobile.Infrastructure.ViewModels.CustomViewModelBase.StartTheMove (Mobile.Entity.BEMoveDetail beMoveDetail, System.Boolean isForeGroundMode) [0x003cf] in /Users/Source/Mobile/Infrastructure/ViewModels/ViewModelBase.cs:419
[MonoDroid]
at Mobile.ViewModel.Move.MoveDetailViewModel.StartMoveDetail (System.Boolean isAlreadyConfirmed) [0x0035a] in /Users/Source/Mobile/ViewModel/Move/MoveDetailViewModel.cs:1533
[MonoDroid]
at Mobile.ViewModel.Move.MoveDetailViewModel.OnPushed () [0x00116] in /Users/Source/Mobile/ViewModel/Move/MoveDetailViewModel.cs:130
[MonoDroid]
at System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.b__7_0 (System.Object state) [0x00000] in <19853c43ab794d18ab1a33ecb65b3c4d>:0
[MonoDroid]
at Android.App.SyncContext+<>c__DisplayClass2_0.b__0 () [0x00000] in <8c07a09624c14764b43f6b946a5a1f23>:0
[MonoDroid]
at Java.Lang.Thread+RunnableImplementor.Run () [0x00008] in <8c07a09624c14764b43f6b946a5a1f23>:0
[MonoDroid]
at Java.Lang.IRunnableInvoker.n_Run (System.IntPtr jnienv, System.IntPtr native__this) [0x00009] in <8c07a09624c14764b43f6b946a5a1f23>:0
[MonoDroid]
at (wrapper dynamic-method).(wrapper dynamic-method).(wrapper dynamic-method) Android.Runtime.DynamicMethodNameCounter.28(intptr,intptr)

Hope these helps.

[vismein]

Hi Mark,
I was able to reproduce this consistently and the exception is thrown in the below line of code
protected Collection<T,TKey,TIndex> GetCollection() { var collection = GetFileCollection().Collection<T, TKey, TIndex>(Name, KeyPredicate); return collection; }

GetFileCollection() returns a vaild object, but the Collection<T, TKey, TIndex>(Name, KeyPredicate) returns this exception.

Thank you for your time.
Regards
Vishnu.

[markmeeus]

Hi Vishnu,

What is really curious is that both stacktraces seem to be having the MarcelloDB.Collections.Collection<T, TKey>.Initialize() method in them.

This shouldn't really not be happening, because they are cached in the collectionFile. Also collectionFiles are cached within the session.

Are you disposing / recreating sessions somewhere?
Can I see the code of GetCollectionFile() ?

[markmeeus]

Could you also post the implementation of GetSession() method?
I have a suspicion something is going on there.
Weren't you having these strange object disposed execptions before? They may all be related...

[vismein]

Hello Mark,

You are right. We had the objectdisposed exceptions in the earlier stages of application development and it got resolved with marcellodb v 1.0.6. We also implemented all your recommendations during that stage and most of the issues got resolved. After 1.0.6, there was a situation when we were iterating a ienumerator and converting to a list caused objectdisposedexception.
So we decided to update the packages to 1.0.8 and we are testing the same at the moment. On further analysis, we could not see any objectdisposedexceptions in the logs, except the one i have reported here (cannot read Int64 to bool).

protected virtual CollectionFile GetFileCollection() {
	var file = GetSession()[$ "{Name}.dat"];
	return file;
}
protected virtual Session GetSession() {
	return _platformFactory.GetSession();
}
public Session GetSession() {
	if (SessionLazy != null) return SessionLazy;
	lock(_thisLock) {
		if (SessionLazy == null) SessionLazy = CreateSession();
		return SessionLazy;
	}
}
private static Session CreateSession() {
	var platform = new Platform();
	var documentsPath = Environment.GetFolderPath(Environment.SpecialFolder.Personal);

	var path = Path.Combine(documentsPath, DbName);

	var directory = new DirectoryInfo(path);
	if (!directory.Exists) directory.Create();

	Debug.WriteLine("********** MARCELLODB *********** Session created.");

	return new Session(platform, path);
}

Hope these helps in your investigation.

[markmeeus]

Is the app able to recover after a restart, or is the data file corrupted somehow?

[markmeeus]

Your info is really helpful btw!

Your code seems ok, even if (I can't see wher it is referenced from, or if it's static) somehow this LazySession got garbage collected, it should not pose any problems.

When you call the Collection<> method for a given name, MarcelloDB should only create the Collection instance once, the first time you call it for that name. On subsequent calls, MarcelloDB uses a cached instance.
The stack trace shows that it doesn't use the cached version, if this would somehow happen a second time, the current behaviour is definitely possible.

Could you confirm (or refute) that this error only happens the first time you open this specific collection? (You could set a collectionFetchedOk boolean and inspect that when the error occurs for instance).

Also,
It would be great if you could grab a screenshot with the collection file expanded, and within that the collections expanded? We should be able to see the cached collections there.