Skip to content

api: clean up failed encoder plugin initialization - #1904

Open
junkilee80 wants to merge 1 commit into
strukturag:masterfrom
junkilee80:fix/encoder-plugin-init-cleanup
Open

api: clean up failed encoder plugin initialization#1904
junkilee80 wants to merge 1 commit into
strukturag:masterfrom
junkilee80:fix/encoder-plugin-init-cleanup

Conversation

@junkilee80

Copy link
Copy Markdown

Both heif_context_get_encoder() variants store a new encoder wrapper in the
output parameter before the plugin has finished initializing it. If the
plugin's new_encoder() callback returns an error, the wrapper remains
allocated and the caller receives a non-null encoder from a failed call. Any
plugin state created before the error can be left behind as well.

Keep the wrapper local until initialization succeeds. On failure, delete it so
the existing plugin cleanup runs, set the output to null, and return the
original error unchanged.

The new test uses an encoder plugin that allocates state and then fails. It
checks both public APIs, the plugin's free callback, and the returned pointer.
The test fails 4 of 8 checks on the old code and passes with the patch, including
under ASan.

Local CTest result: 51 passed, 15 skipped. The existing region test still
fails in this minimal-codec build; the same seven assertions fail on unmodified
master with the same configuration.

Found while reviewing RepoAudit output. PailGen suggested the cleanup pattern;
I reviewed the ownership path and added the regression test and validation.

Signed-off-by: Junki Lee <junkilee80@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant