Bug 692340 - Mupdf runs out of memory with large PDF file on android
Summary: Mupdf runs out of memory with large PDF file on android
Status: RESOLVED FIXED
Alias: None
Product: MuPDF
Classification: Unclassified
Component: mupdf (show other bugs)
Version: unspecified
Hardware: Macintosh MacOS X
: P4 normal
Assignee: Robin Watts
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-07-11 19:04 UTC by William Lee
Modified: 2011-12-30 12:41 UTC (History)
1 user (show)

See Also:
Customer:
Word Size: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description William Lee 2011-07-11 19:04:58 UTC
Hi there, 
I am an android developer and i am working on an app which require to read large PDF file( 300mb) I used the mupdf but the app crash due to the memory warning. I am able to read 2 pages but when I turn to page 3, the app crashed. Is there anyway that I can release those pages which is not showing and only draw single page and read single page's informaton to save memory?
Comment 1 Robin Watts 2011-07-12 12:05:05 UTC
We already release the page contents; we may keep the file structure around.

See comment #5 on bug 692278 for a users suggestion. Does that fix it for you?

This is untested at this time.
Comment 2 William Lee 2011-07-12 23:28:44 UTC
(In reply to comment #1)
> We already release the page contents; we may keep the file structure around.
> 
> See comment #5 on bug 692278 for a users suggestion. Does that fix it for you?
> 
> This is untested at this time.

The trick he had does not fix the problem. The app still crashed when I try to open page 3. Are there additional trick I need to do in MuPdfActivity.java?
Comment 3 Robin Watts 2011-07-14 18:49:56 UTC
It maybe that page 3 of this file simply uses too much memory for us to be able to view it. Can you try altering the sources so we jump straight in at page 3?

Does Page 3 have a different sort of content than previous pages?

MuPDF normally assumes that allocations aren't going to fail - we could do with spending some time making it cope better with out of memory conditions. Unfortunately in the absence of a customer request, this may not happen in the immediate future.
Comment 4 Robin Watts 2011-07-21 15:57:44 UTC
William: One possible way to proceed. Could you please use pdfclean (part of mupdf) on windows or unix to extract us just page 3 of your example file.

Presumably android mupdf will then crash when given just that file?

If so, could you attach that file to this bug please? It would allow us to reproduce the problem locally and we may be able to solve it.
Comment 5 William Lee 2011-07-21 20:41:09 UTC
(In reply to comment #4)
> William: One possible way to proceed. Could you please use pdfclean (part of
> mupdf) on windows or unix to extract us just page 3 of your example file.
> 
> Presumably android mupdf will then crash when given just that file?
> 
> If so, could you attach that file to this bug please? It would allow us to
> reproduce the problem locally and we may be able to solve it.


Sorry I can not provide the pdf but I can provide the log. the following log shows that I open the pdf first (so page 0 opened) then I clicked go to the last page (so jump to page 290) then I click go up one page again (go to page 289) Then the app crashed. Hopefully the log will help

E/libmupdf(  245): Opening document...
E/libmupdf(  245): Loading page tree...
E/libmupdf(  245): Done! 291 pages
D/dalvikvm(  245): Trying to load lib /data/data/com.artifex.mupdf/lib/libmupdf.so 0x44ede278
D/dalvikvm(  245): Shared lib '/data/data/com.artifex.mupdf/lib/libmupdf.so' already loaded in same CL 0x44ede278
I/System.out(  245): PixmapView construct
E/libmupdf(  245): Goto page 0...
I/ActivityManager(   59): Displayed activity com.artifex.mupdf/.MuPDFActivity: 3795 ms (total 3795 ms)
E/libmupdf(  245): PageWidth=1224
E/libmupdf(  245): PageHeight=1584
I/System.out(  245): scalePageToScreen: Raw=1224.0x1584.0 scaled=480,621 pageScale=0.39215687
I/System.out(  245): page=480,621 (1224.0,39215687">1584.0@0.39215687) @ 0,34
I/System.out(  245): patch=[0,0,480,621]
D/dalvikvm(  245): GC_EXTERNAL_ALLOC freed 992 objects / 68872 bytes in 85ms
I/System.out(  245): Calling redraw native method
I/libmupdf(  245): In native method
I/libmupdf(  245): Checking format
I/libmupdf(  245): locking pixels
E/libmupdf(  245): Rendering page=480x621 patch=[0,0,480,621]
D/dalvikvm(  117): GC_EXPLICIT freed 185 objects / 9624 bytes in 1470ms
E/libmupdf(  245): Rendered
I/System.out(  245): Called native method
I/System.out(  245): Blitting bitmap from 0,0,480,621 to 0.0,34.0,480.0,655.0
I/System.out(  245): Render thread sleeping
I/System.out(  245): Render thread woken
E/libmupdf(  245): Goto page 0...
E/libmupdf(  245): PageWidth=1224
E/libmupdf(  245): PageHeight=1584
I/System.out(  245): scalePageToScreen: Raw=1224.0x1584.0 scaled=480,621 pageScale=0.39215687
I/System.out(  245): page=480,621 (1224.0,39215687">1584.0@0.39215687) @ 0,34
I/System.out(  245): patch=[0,0,480,621]
I/System.out(  245): Calling redraw native method
I/libmupdf(  245): In native method
I/libmupdf(  245): Checking format
I/libmupdf(  245): locking pixels
E/libmupdf(  245): Rendering page=480x621 patch=[0,0,480,621]
E/libmupdf(  245): Rendered
I/System.out(  245): Called native method
I/System.out(  245): Blitting bitmap from 0,0,480,621 to 0.0,34.0,480.0,655.0
I/System.out(  245): Render thread sleeping
I/System.out(  245): Render thread woken
E/libmupdf(  245): Goto page 290...
I/ActivityManager(   59): Process com.android.email (pid 227) has died.
I/ActivityManager(   59): Process android.process.acore (pid 147) has died.
I/ActivityManager(   59): Process com.android.quicksearchbox (pid 193) has died.
I/ActivityManager(   59): Process com.android.protips (pid 187) has died.
I/ActivityManager(   59): Process com.android.music (pid 176) has died.
E/libmupdf(  245): PageWidth=1224
E/libmupdf(  245): PageHeight=1584
I/System.out(  245): scalePageToScreen: Raw=1224.0x1584.0 scaled=480,621 pageScale=0.39215687
I/System.out(  245): page=480,621 (1224.0,39215687">1584.0@0.39215687) @ 0,34
I/System.out(  245): patch=[0,0,480,621]
I/System.out(  245): Calling redraw native method
I/libmupdf(  245): In native method
I/libmupdf(  245): Checking format
I/libmupdf(  245): locking pixels
E/libmupdf(  245): Rendering page=480x621 patch=[0,0,480,621]
E/libmupdf(  245): Rendered
I/System.out(  245): Called native method
I/System.out(  245): Blitting bitmap from 0,0,480,621 to 0.0,34.0,480.0,655.0
I/System.out(  245): Render thread sleeping
I/System.out(  245): Render thread woken
E/libmupdf(  245): Goto page 289...
I/ActivityManager(   59): Process android.process.media (pid 205) has died.
I/ActivityManager(   59): Process com.android.alarmclock (pid 167) has died.
I/ActivityManager(   59): Low Memory: No more background processes.
I/ActivityManager(   59): Process com.android.launcher (pid 117) has died.
I/WindowManager(   59): WIN DEATH: Window{44fe38c0 com.android.launcher/com.android.launcher2.Launcher paused=false}
I/ActivityManager(   59): Low Memory: No more background processes.
I/WindowManager(   59): WIN DEATH: Window{4506c250 SurfaceView paused=false}
I/ActivityManager(   59): Process com.android.inputmethod.latin (pid 110) has died.
W/ActivityManager(   59): Scheduling restart of crashed service com.android.inputmethod.latin/.LatinIME in 5000ms
I/WindowManager(   59): WIN DEATH: Window{4506b390 com.artifex.mupdf/com.artifex.mupdf.MuPDFActivity paused=false}
I/ActivityManager(   59): Low Memory: No more background processes.
I/ActivityManager(   59): Process com.artifex.mupdf (pid 245) has died.
D/skia    (  114): purging 6K from font cache [1 entries]
I/ActivityManager(   59): Start proc com.android.launcher for activity com.android.launcher/com.android.launcher2.Launcher: pid=253 uid=10017 gids={}
I/ActivityManager(   59): Low Memory: No more background processes.
D/dalvikvm(  114): GC_EXPLICIT freed 4124 objects / 223688 bytes in 335ms
I/UsageStats(   59): Unexpected resume of com.android.launcher while already resumed in com.artifex.mupdf
I/ActivityThread(  253): Publishing provider com.android.launcher2.settings: com.android.launcher2.LauncherProvider
D/dalvikvm(  253): GC_EXTERNAL_ALLOC freed 983 objects / 73208 bytes in 83ms
W/InputManagerService(   59): Got RemoteException sending setActive(false) notification to pid 245 uid 10036
I/ActivityManager(   59): Displayed activity com.android.launcher/com.android.launcher2.Launcher: 1685 ms (total 1685 ms)
D/dalvikvm(  253): GC_EXTERNAL_ALLOC freed 11351 objects / 575904 bytes in 93ms
D/dalvikvm(  253): GC_EXPLICIT freed 1109 objects / 56936 bytes in 74ms
I/ActivityManager(   59): Start proc com.android.inputmethod.latin for service com.android.inputmethod.latin/.LatinIME: pid=261 uid=10018 gids={}
I/ActivityManager(   59): Start proc android.process.acore for content provider com.android.providers.userdictionary/.UserDictionaryProvider: pid=269 uid=10000 gids={3003, 1015}
I/ActivityThread(  269): Publishing provider com.android.social: com.android.providers.contacts.SocialProvider
I/ActivityThread(  269): Publishing provider applications: com.android.providers.applications.ApplicationsProvider
I/ActivityThread(  269): Publishing provider contacts;com.android.contacts: com.android.providers.contacts.ContactsProvider2
I/ActivityThread(  269): Publishing provider call_log: com.android.providers.contacts.CallLogProvider
I/ActivityThread(  269): Publishing provider user_dictionary: com.android.providers.userdictionary.UserDictionaryProvider
D/dalvikvm(   59): GC_EXPLICIT freed 11563 objects / 554368 bytes in 165ms
D/skia    (   59): purging 235K from font cache [25 entries]
D/dalvikvm(   59): GC_EXPLICIT freed 2824 objects / 139200 bytes in 98ms
D/SntpClient(   59): request time failed: java.net.SocketException: Address family not supported by protocol
Comment 6 Robin Watts 2011-07-21 23:41:10 UTC
(In reply to comment #5)
> Sorry I can not provide the pdf but I can provide the log. the following log
> shows that I open the pdf first (so page 0 opened) then I clicked go to the
> last page (so jump to page 290) then I click go up one page again (go to page
> 289) Then the app crashed. Hopefully the log will help

Sadly, no. All it tells me is that it's running short of memory and then dying. Without a PDF to show the problem, it's going to be hard to reproduce it.

Can you at least tell me what the content of the PDF is? Text? Line Art? Images? The more details the better...
Comment 7 William Lee 2011-07-21 23:49:19 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > Sorry I can not provide the pdf but I can provide the log. the following log
> > shows that I open the pdf first (so page 0 opened) then I clicked go to the
> > last page (so jump to page 290) then I click go up one page again (go to page
> > 289) Then the app crashed. Hopefully the log will help
> 
> Sadly, no. All it tells me is that it's running short of memory and then dying.
> Without a PDF to show the problem, it's going to be hard to reproduce it.
> 
> Can you at least tell me what the content of the PDF is? Text? Line Art?
> Images? The more details the better...

The pdf contains high resolution image with internal links to different pages. Each pages is about 1Mb in size (so it adds up to 300 Mb). Hope this information helps.
Comment 8 Robin Watts 2011-12-21 14:46:16 UTC
We have a new branch of development in git, the 'context' branch. That should hopefully solve this problem...
Comment 9 Robin Watts 2011-12-30 12:41:41 UTC
This should be solved by recent reworkings of the store.

We now keep objects in a 'store' once we have finished with them, so we can quickly reuse them without redecoding. If memory gets low, however, we 'scavenge' memory from the store, discarding them. This should stop us running out of memory unnecessarily.