In gs (and mupdf) we have our own memory allocation functions; we would like all the libraries we call to use these function rather than call vanilla malloc/free. Problem 1: There is no way to stop OpenJPEG calling malloc and free directly. This can be solved with a (fairly trivial) tweak to opj_malloc.h so that when built with a suitable predefine (OPJ_USER_MALLOC or something similar), calls to opj_malloc etc are NOT #defined down to malloc, and versions of opj_malloc can be supplied by the user. This would be a fairly simple and harmless patch that I hope we could persuade the OpenJPEG developers to take on, but it might be problematic for shared library builds of OpenJPEG. Problem 2: If user supplied allocators are to be called, then it's probable that those allocators would need to take some sort of argument (in gs, they would need a gs_memory_t, and in mupdf an fz_context *). There is no scope to add this without breaking the OpenJPEG API. Some discussion therefore needs to take place with the OpenJPEG developers to try to see if they are amenable to some sort of solution for this, together with the API changes it would require.
Bountiable to Shelly as he has a relationship with the openjpeg developers, I believe.
I emailed an outline of this proposal to my contact on the 2/Oct/2013 but have not heard back yet. I will send a reminder to them around the 16/Oct.
I nudged the OpenJPEG2 team for an update and got the following response :- "No, nothing done on openjpeg lately sorry :( And I cannot even give you a clear vision of my agenda in the coming months... I am CCing the team in case someone else has some time in the coming weeks." Personally, I think our best bet right now is to map out our possible changes and ask them to review it, set a time limit for a response and then continue preferably with but possibly without their feedback. Do you guys have any thoughts on this? Should I continue looking into this or do one of you want to step in to add some weight to the discussions?
Analysis of the existing OpenJPEG2 code has highlighted a number of issues :- 1) The main context (opj_codec_private_t) is defined in openjpeg.c and hence hidden from the rest of the source files. A "void *" is used throughout to access the context but all function calls that need to know the internal structure pass through openjpeg.c. 2) The OpenJPEG2 memory allocation calls make no provision for a pointer to our internal memory. 3) A number of the memory allocation calls are nested very deep and nowhere near the high level context. To overcome these problems will take a number of staged updates which the OpenJPEG2 team may be more likely to accept for merging in. The stages should include :- 1) Make opj_codec_private_t available through an include file. 2) Propagate the main context to the lower function calls. 3) Add a memory pointer to the main context for our internal memory. 4) Make all memory function calls access our internal memory. Stage 1 has been submitted to OpenJPEG2 for review and I will update this bug when I hear back from them.
While this approach would provide a way to set memory function callouts: > 1) Make opj_codec_private_t available through an include file. If the OpenJPEG developers object, then providing a function that sets the memory callouts (alloc, free, realloc, ...) would suffice as well, correct ? (similar to gsapi_set_stdio). Of course, all of the memory callout functions must pass a 'void *' which the provider functions can use, as well as the 'standard' parameters. Just a suggestion.
I have not had a reply from the OpenJPEG2 team so I emailed them again on the 2/Dec asking for an update.
The OpenJPEG team requested that this bug is opened in their own system. The first stage changes can now be tracked at https://code.google.com/p/openjpeg/issues/detail?id=253
(In reply to comment #7) > The OpenJPEG team requested that this bug is opened in their own system. The > first stage changes can now be tracked at > https://code.google.com/p/openjpeg/issues/detail?id=253 There was a response 1/13/2014 which needs a follow up, let me know if you are going to do that, Shelly. Thanks.
The openjpeg team are in the process of making OpenJPEG 2.1.0 and I have updated their bug report today with more details.
The first patch has been accepted by the OpenJPEG team. I will now progress onto the second patch.
Created attachment 10990 [details] Add GS memory management to OpenJPEG2.1 The second patch has now been submitted under a new bug report (https://code.google.com/p/openjpeg/issues/detail?id=356) to the OpenJPEG team. This has been regression tested and checked into git (1415c32878bcdaf6b9ec83718c34e50607d0eff2).
I contacted the OpenJPEG team for an update and they have confirmed that this bug has now changed status to "accepted" and been upgraded from medium to high priority.
The OpenJPEG team contacted me on the 17th July with concerns about the depth of changes needed for the memory management. I sent them an update on the 21st July but got no response. I sent another email on the 18th of Aug but still got no response. I then sent another email on the 26th Aug but have not heard anything yet. I will wait a few more days and then email them again. Should I emphasise that we are considering forking the code if we do not hear back from them soon?
The OpenJPEG team have provided 2 alternative patches for this work and I am currently reviewing them.
The updated patch from the OpenJPEG team has been regression tested and worked fine. I have informed them of the test results and await their decision.
The patch from the OpenJPEG team has been regression tested and highlighted that they have not implemented a bug fix sent to them by us in https://code.google.com/p/openjpeg/issues/detail?id=235 Without this patch a number of the files show slightly blurred images compared to our main HEAD. I have updated the OpenJPEG team and await their reply.
I have nudged the OpenJPEG team again today to try and get some movement on this bug.
The OpenJPEG team requested that I fork their GitHub code and make a pull request. I have updated the code, regression tested it in Ghostscript and made the pull request today.
Hi Shelly, what is the status of this bug? It looks like gs is not using malloc/free for openjpeg now but it looks like there was more to this.
We also noticed a global variable "opj_memory", not sure if that is workaround until the openjpeg group can make changes in their code. If it is not going to go away soon we'll add a mutex.
As of May 2017 they had mainly accepted the patch which worked with the new memory manager code and the existing API but were still requesting more changes with no commitment to actually include it. (See https://github.com/uclouvain/openjpeg/pull/582 ) However, the OpenJpeg team wanted to know what would happen if someone switched between the old and new API part way through processing a file! They boiled it down to either having both API present for a short while or going the full way and switching over to the new one. Bottom line is they only want to do a full change over and ditch the old API but they cant justify moving to the new one! Not sure how we progress from here.