Memory use of this command is much larger for normal builds than luratech builds: gs -sOutputFile=/dev/null -dBATCH -dNOPAUSE -r300 -sDEVICE=ppmraw ../tests_private/comparefiles/Bug694621.pdf In a normal build, this has maximum heap usage (from valgrind --tool=massif) of 150MB, of which 109,511,619B is from opj_tcd_decode_tile() where it allocates a UINT32 for each pixel. Luratech's maximum heap usage for the same input file is 40MB. It looks like luratech's equivalent to opj_tcd_decode_tile() is s_jpxd_process() which allocates 27,377,904B, almost exactly a quarter of normal build. Luratech's code does not default to 32 bits per pixel. Instead it appears to explicitly look at the colorspace and allocate accordingly. In the above case, we end up with 8 bits per pixel, hence reducing memory use by a factor of 4. The callback s_jpxd_write_data() uses colorspace info when generating the output data. So it might be worth improving opj_tcd_decode_tile() to use a similar technique to luratech, in order to avoid the excessive memory use for this sort of input file.
*** Bug 702095 has been marked as a duplicate of this bug. ***
*** Bug 702094 has been marked as a duplicate of this bug. ***
*** Bug 702093 has been marked as a duplicate of this bug. ***
*** Bug 702092 has been marked as a duplicate of this bug. ***
*** Bug 702091 has been marked as a duplicate of this bug. ***
Assigning to Henry to determine who should do this (and when), possibly after discussion at an engineering meeting.
Why not request an enhancement upstream?
For us to do this optimization would be a fair bit of work. I have reported this upstream, let's see what they say: https://github.com/uclouvain/openjpeg/issues/1252
> let's see what they say: Their immediate response was as follows: "Intermediate computation need more than 8 bits. Probably 16 bits is enough, but some research in JPEG2000 reference literature should be done to confirm it. And OpenJPEG supports also 16-bit images, which would thus needs 32 bits. Not to mention irreversible which use float32 intermediate computations. So we'd need to have two code paths, or use some templating code (switching to C++) to avoid hugly macros. That would be a really non trivial effort !" Julian remarked that it is surprising that they don't think more than 32bpp intermediate results are necessary for 32bpp images.