Bug 692162

Summary:	res_pixmap.c thread issue
Product:	MuPDF	Reporter:	Pedro Rivera <pedro.rivera.651>
Component:	fitz	Assignee:	Tor Andersson <tor.andersson>
Status:	RESOLVED DUPLICATE
Severity:	normal	CC:	danwaleke, zeniko
Priority:	P4
Version:	unspecified
Hardware:	All
OS:	All
Customer:		Word Size:	---

Description Pedro Rivera 2011-04-19 21:56:12 UTC

The variables fz_memory_limit and fz_memory_used are not thread safe. I am assuming that these were setup with the idea of managing memory for a single document / single page at a time. This is great for simple desktop viewers but bad for heavy server side multi-threaded applications.

When I tested multi-threaded page rendering i started getting a lot of "memory limits exceeded warnings" almost immediately. The result was that many pages did not get fully rendered. 

Adding TLS to the variables seems to give the desired behavior when implementing executor services in Java.

  static int __thread fz_memory_limit = 256 << 20;
  static int __thread fz_memory_used = 0;

Thanks,

Pedro

Comment 1 zeniko 2011-04-19 22:20:05 UTC

Ignoring the fact that the code indeed isn't thread safe, this looks rather like a case of too low a limit for your case. IIRC, the limit is supposed to be global (after all, your computer doesn't get more memory when rendering more documents in parallel) - so you'll rather want to make it match what your machine can reasonably handle.

Also, not every MuPDF consumer will allocate images on the same thread as they're released (we don't), so per-thread variables aren't a general enough solution.

As with the static FreeType references, we'd need a context struct that contains these values and where we can control ourselves whether it applies per thread, per document or globally (with locking in place in our code).

Comment 2 Dan 2011-06-07 21:48:23 UTC

What are the chances of making MuPDF thread safe?  

I have wrapped the library in a simple win32 DLL that can be PInvoked from managed c# code.  Occastionally the code is called by different threads in the application.  The application crashes because of the global static FreeType reference.

I have changed this in my code, but I am no c developer.  I'd prefer if someone that maintains the library could make this a priority.

Side note: I love mupdf.  Just wish it was thread safe.

Comment 3 Pedro Rivera 2011-06-07 22:58:11 UTC

(In reply to comment #2)

Dan,

If you are doing C# you might want to look at JMuPDF which is a Java wrapper around MUPDF toolkit. This project already addresses several thread issues. With some minor changes I bet you can get a C# version going. The project is on SourceForge. Just a thought. Good luck.

Pedro

> What are the chances of making MuPDF thread safe?  
> 
> I have wrapped the library in a simple win32 DLL that can be PInvoked from
> managed c# code.  Occastionally the code is called by different threads in the
> application.  The application crashes because of the global static FreeType
> reference.
> 
> I have changed this in my code, but I am no c developer.  I'd prefer if someone
> that maintains the library could make this a priority.
> 
> Side note: I love mupdf.  Just wish it was thread safe.

Comment 4 Pedro Rivera 2011-08-23 01:03:16 UTC


*** This bug has been marked as a duplicate of bug 692023 ***