Out of memory error while using templates

[SmithaRSanthosh]

I am using docx4j to generate word and ppt reports from a web application. We are using both create new report and loading a template and creating a report approach for creating the reports. While using the templates we find that the server throws an out of memory error very often.
We load the template using

Code: Select all

PresentationMLPackage pMLPackage =
         (PresentationMLPackage)OpcPackage.load(new java.io.File(inputfilepath));

and save it finally using

Code: Select all

pMLPackage.save(new File(System.getProperty("user.dir") + "/Outputs/BA_Multiple_INS_OUT.pptx"));

We do not close or assign the pMLPackage to null anywhere. Can you help us with the issue?

[jason]

Without a clearer explanation of when and how you create your package objects, it is hard to go beyond general advice.

Do you think there is a memory leak? (How many documents are being processed concurrently, and how big are they? Is the memory available again after processing in complete?)

Are you storing the pkg in app server session objects? How long to the sessions last? How many are there?

What are your VM memory settings? is it enough for the projected workload?

We do not close or assign the pMLPackage to null anywhere.

Are you reusing it? How many of them are hanging around (see above)

[SmithaRSanthosh]

We are using an input from the UI to create and open a word file. On click of an UI button the user is prompted if he wants to save /open the file.

We use the below code to accomplish create and open of a file. Here we create a new WordprocessingMLPackage . In the method generateReachViewDistVennDiagram we are loading another docx file to the package named vennWordMLPackage. We are copying the contents of the vennWordMLPackage to the original word package. We are then attaching the file to the response object.

Syntax: [ Download ] [ Hide ]

Syntax: [ Download ] [ Show ]

Using java Syntax Highlighting

public class WordGenerator {
               private WordprocessingMLPackage wordMLPackage;
private ObjectFactory factory;


private WordprocessingMLPackage vennWordMLPackage;

public void export(HttpServletResponse res, Bhvr obAnalysisExport,
                                        ExportParam obExportParam) throws Exception {
                                       
        if (this.wordMLPackage == null) {
                initializeExportSettings();
        }                              
        generateReachViewDistVennDiagram() ;
        attachFileToResponse(res);
}



private void initializeExportSettings() throws InvalidFormatException {
        this.wordMLPackage = WordprocessingMLPackage.createPackage();
        this.factory = new ObjectFactory();    
}

private void generateReachViewDistVennDiagram() throws Exception {
        // Loading the stored templates of Venn Diagram
        if(vennWordMLPackage == null) {
                vennWordMLPackage = WordprocessingMLPackage.load(new File("resources/VennDiagram.docx"));
        }
        createWordMLPackageFootnote(reachExport.getVennChartFootNote(), reachExport.getVennChartTitle(), true, 20);

        List<Object> vennContents = vennWordMLPackage.getMainDocumentPart().getContent();
        List<Object> vennContentsCopy = new ArrayList<Object>(vennContents.size());
        // Creating copy of the Venn Template elements to avoid duplicate objects in multiple insights export.
        for (Object content : vennContents) {
                Object newObj = XmlUtils.deepCopy(content);
                wordMLPackage.getMainDocumentPart().getContent().add(newObj);
                vennContentsCopy.add(newObj);
        }
       
}
       
private void attachFileToResponse(HttpServletResponse response)
                                        throws Exception {
        File file = exportHelper.generateTemporaryFile("docx");
        ServletOutputStream stream = null;
        BufferedInputStream buf = null;
        try {
                stream = response.getOutputStream();
                response.setContentLength((int) file.length());
                FileInputStream input = new FileInputStream(file);
                buf = new BufferedInputStream(input);
                int readBytes = 0;
                while ((readBytes = buf.read()) != -1)
                        stream.write(readBytes);
        } catch (IOException ioe) {
                throw new ServletException(ioe.getMessage());
        } catch (Exception e) {
                throw e;
        } finally {
                if (stream != null)
                        stream.close();
                if (buf != null)
                        buf.close();
                file.delete();
                vennWordMLPackage = null;
                factory = null;
                endnotes = null;
                bhvrExport = null;
                exportParam = null;
                reachExport = null;
                exportHelper = null;
        }
}
}

Parsed in 0.019 seconds, using GeSHi 1.0.8.4

We do have a scenario where we will have to process the vennWordMLPackage multiple times. In that case , we will create one wordMLPackage and then call the generateReachViewDistVennDiagram method multiple times. We will load the vennWordMLPackage multiple times and keep adding each iteration content to the original wordMLPackage. However we receive the out of memory even when we did not use this scenario. We received the out of memry when we Hit this code a few times.
We are not saving the package in session but we are attaching it to response and opening / saving the file on the clients browser.

[SmithaRSanthosh]

We also used profiling to find out the objects in memory at any time and found these object even when we are not using docx4j related links.

org.docx4j 1 16 bytes
org.docx4j.bibliography 33 792 bytes
org.docx4j.customXmlProperties 3 48 bytes
org.docx4j.customxml 4 64 bytes
org.docx4j.dml 1,756 40,160 bytes
org.docx4j.dml.chart 814 14,744 bytes
org.docx4j.dml.chartDrawing 54 864 bytes
org.docx4j.dml.diagram 472 10,760 bytes
org.docx4j.dml.diagram2008 18 288 bytes
org.docx4j.dml.picture 5 80 bytes
org.docx4j.dml.spreadsheetdrawing 74 1,224 bytes
org.docx4j.dml.wordprocessingDrawing 107 2,112 bytes
org.docx4j.docProps.core 24 384 bytes
org.docx4j.docProps.core.dc.elements 2 32 bytes
org.docx4j.docProps.core.dc.terms 1 16 bytes
org.docx4j.docProps.custom 67 1,064 bytes
org.docx4j.docProps.extended 55 880 bytes
org.docx4j.docProps.variantTypes 148 2,336 bytes
org.docx4j.math 236 4,272 bytes
org.docx4j.model.structure 12 312 bytes
org.docx4j.model.structure.jaxb 3 48 bytes
org.docx4j.openpackaging.contenttype 6 88 bytes
org.docx4j.openpackaging.parts 3 48 bytes
org.docx4j.openpackaging.parts.relationships 4 96 bytes
org.docx4j.relationships 5 80 bytes
org.docx4j.vml 837 14,200 bytes
org.docx4j.vml.officedrawing 286 5,584 bytes
org.docx4j.vml.presentationDrawing 1 16 bytes
org.docx4j.vml.root 1 16 bytes
org.docx4j.vml.spreadsheetDrawing 30 848 bytes
org.docx4j.vml.wordprocessingDrawing 67 1,824 bytes
org.docx4j.wml 1,878 40,208 bytes
org.docx4j.xmlPackage 12 192 bytes

We also do get an exception when executing the docx4j code. This is related to not finding the docx4j.properties file. Our concern here is that this results in a null pointer exception as follows:

Code: Select all

ava.lang.NullPointerException
   at org.docx4j.utils.ResourceUtils.getResource(ResourceUtils.java:45)
   at org.docx4j.Docx4jProperties.init(Docx4jProperties.java:19)
   at org.docx4j.Docx4jProperties.getProperties(Docx4jProperties.java:34)
   at org.docx4j.openpackaging.io.SaveToZipFile.saveRawXmlPart(SaveToZipFile.java:207)
   at org.docx4j.openpackaging.io.SaveToZipFile.saveRawXmlPart(SaveToZipFile.java:194)
   at org.docx4j.openpackaging.io.SaveToZipFile.savePart(SaveToZipFile.java:410)
   at org.docx4j.openpackaging.io.SaveToZipFile.addPartsFromRelationships(SaveToZipFile.java:373)
   at org.docx4j.openpackaging.io.SaveToZipFile.save(SaveToZipFile.java:164)
   at org.docx4j.openpackaging.io.SaveToZipFile.save(SaveToZipFile.java:105

The concern is that the SaveToZipFile.saveRawXmlPart also has code to close output streams . Will these output streams remain open due to this exception and therby result in memory issues?

[jason]

You could add docx4j.properties to your path to eliminate that concern.

The things you quote are packages. Is the next number the number of objects in that package? Maybe they just haven't been garbage collected yet?

[SmithaRSanthosh]

I added docx4j.properties to the classes folder and the Web-Inf folders, but it still throws the same exception.

Again the list I had attached was the package names of objects in memory and Instance count and the Size . Was concerned about so many docx4j objects in memory and if there is any relation between this and the out of memory error that kept occuring.

[jason]

No, those are package names. Packages contain classes.

As I asked before, how large are the documents you are processing (in KB, and number of pages)?

As I asked before, what are your JVM memory settings?