bug: Memory leak using Dart_PostCObject because need to consider the return value is true or false

[fzyzcjy]

Hi thanks for the lib! I suspect there is memory leak when using Dart_PostCObject. Please refer to: dart-lang/sdk#47270 (comment)

[shekohex]

Is this still an issue?

[fzyzcjy]

@shekohex Yes. allo-isolate does not handle the case when returning false. please see dart-lang/sdk#47270 (comment) and dart-lang/sdk#47270 (comment) (he mentioned he changed the code in the first post)

[shekohex]

In allo-isolate we do free the memory regardless of the Dart_PostCObject return value, this the current behavior here:
https://github.com/sunshine-protocol/allo-isolate/blob/51caf0367552bd290ce8b90c62b19db913e68c64/src/lib.rs#L119-L129

Okay, Let's take a deep dive a bit into the VM code.

Where is the Dart_PostCObject?
You will see it is defined here:
https://github.com/dart-lang/sdk/blob/a3f8a9562a8d98be7f3b1cc42bb89854f5c77c90/runtime/vm/native_api_impl.cc#L59-L61 This function does not do a lot on itself, it calls inner function here: https://github.com/dart-lang/sdk/blob/a3f8a9562a8d98be7f3b1cc42bb89854f5c77c90/runtime/vm/native_api_impl.cc#L46-L57 can only return false in one case if the message points to a null memory.

But this only happens, if the WriteApiMessage returned null which is implemented here: https://github.com/dart-lang/sdk/blob/e995cb5f7cd67d39c1ee4bdbe95c8241db36725f/runtime/vm/message_snapshot.cc#L3772-L3781

and again, this would happen if the ApiMessageSerializer failed to Serialize the message.

So, let's see how the ApiMessageSerializer handles the message!

in the Serialize method here: https://github.com/dart-lang/sdk/blob/e995cb5f7cd67d39c1ee4bdbe95c8241db36725f/runtime/vm/message_snapshot.cc#L3613-L3653 you will find it calls Push with the reference to the message https://github.com/dart-lang/sdk/blob/e995cb5f7cd67d39c1ee4bdbe95c8241db36725f/runtime/vm/message_snapshot.cc#L3066-L3071

And here is the stack_.Add(object) https://github.com/dart-lang/sdk/blob/e995cb5f7cd67d39c1ee4bdbe95c8241db36725f/runtime/vm/message_snapshot.cc#L3068 which I guess do a local copy of the value (not 100% sure yet, but I will keep dive more).

To be continued...

[fzyzcjy]

Thanks for the detailed reply!

To be continued...

Tell me when you finished ;) I will come back and read the rest