Bug 697545 - Fix memory squeezing issues
Summary: Fix memory squeezing issues
Alias: None
Product: Ghostscript
Classification: Unclassified
Component: General (show other bugs)
Version: master
Hardware: PC Windows NT
: P4 normal
Assignee: Shailesh Mistry
Depends on:
Reported: 2017-02-03 03:40 UTC by Robin Watts
Modified: 2023-05-25 07:47 UTC (History)
2 users (show)

See Also:
Word Size: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Robin Watts 2017-02-03 03:40:01 UTC
Memento includes a "fast memory squeezing" feature that allows us to test 'failure paths' in gs/gpcl6/etc for behaviour after malloc exhaustion.

Running this on both gs and gpcl, it's clear that there are problem cases. We'd like to solve these.

To generate a memory squeeze report, run the following on linux:

  MEMENTO_SQUEEZEAT=1 membin/gs -sDEVICE=bit -o /dev/null examples/tiger.eps > log 2>&1

This will take a while to run, and generate a large log. Killing it after a minute or so will easily give you enough to be going on with.

You can either read 'log' yourself, or (for a slightly more readable view), you can run it through the /home/robin/squeeze2html.pl script on casper:

  cat log | perl /home/robin/squeeze2html.pl > squeeze.html

If you'd rather not generate your own reports, you can see a couple of existing ones:


The first of these comes from:

 MEMENTO_SQUEEZEAT=1 membin/gpcl6 -sDEVICE=bit -o /dev/null J11.pxl > log 2>&1

(J11.pxl can be found on casper as /home/regression/tests_private/xl/JEITA/J11.pxl)

The second comes from:

 MEMENTO_SQUEEZEAT=1 membin/gs -sDEVICE=bit -o /dev/null examples/tiger.eps > log 2>&1

In the generated reports, click on each number to show/hide the output given from failing the allocations after the nth one.

Green numbers indicate cases where the application correctly shut down with no leaked allocations. Orange cases show the application shutting down without crashing, but with some leaked blocks. Red cases are where we hit a SEGV. (I like traffic lights).

Ideally everything should be green, but as a first step, getting rid of the red cases would be good.

I had a quick prod at the early failures for PCL, and came up with:




Those fixes were enough to change 1 - 178 to being green. So small fixes like this can make a huge impact on the reports - it's hopefully not the huge daunting task that it might initially look like.

Shelly: If you'd like access to one of the cluster nodes to run the memory squeezer on, please just say. Probably best if we don't run it on casper.

Similarly, if you'd rather not run the squeezer yourself, you can reproduce the failures on Windows in MSVC.

Do a memento build, then put a breakpoint on Memento_breakpoint. Open the build properties and in the environment field put: "MEMENTO_FAILAT=100" (or whatever the actual number is. Run the build, and you'll stop in the debugger where the allocation is about to fail. You can then step manually to watch what happens.

Sadly, there may be some differences in allocation between windows and linux (because the platform dependent code may do some allocations). As such it may be necessary to adjust the exact failure numbers, but it should be a (fairly) constant small offset.

If need be I can regenerate those reports periodically for you. Let me know if you have any questions.
Comment 1 Henry Stiles 2017-02-03 12:33:48 UTC
Hello Shelly, please start with gs and not pcl.  We have a large commit pending and you will likely create conflicts by fixing the PCL problems.
Comment 2 Robin Watts 2017-03-16 09:29:41 UTC
The following commit was added to our repo:

commit 64f5d62c80176085335d84c24e02cd49d9e032b9
Author: Robin Watts <robin.watts@artifex.com>
Date:   Fri Mar 3 19:42:29 2017 +0000

    Bug 697545: Extend "separable_and_linear" processing.

    First, avoid needless check_separable_and_linear tests.

It should have referenced bug 697585, sorry.