Document conversion API in OSGI environment

GroupDocs.Conversion Product Family
1

Hi,

I’m doing an evaluation POC of your product “GroupDocs.Conversion v181.12” in order to use it for converting files to pdf.
The results are really good when running them locally in junit tests.

But we want to use it in a karaf osgi VM. So for my test I installed your bundle by wrapping it (wrap:mvn:com.groupdocs/groupdocs-conversion/18.12). Unfortunatly, all conversion are getting into exception :
Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at com.aspose.words.internal.aM.j(Unknown Source)
at com.aspose.words.afZ.handleText(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xa.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xe.read(Unknown Source)
at com.aspose.words.aa.b(Unknown Source)
at com.aspose.words.aa.a(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.groupdocs.conversion.domain.savers.email.e.c(Unknown Source)
at com.groupdocs.conversion.domain.documents.f.a(Unknown Source)
at com.groupdocs.conversion.operations.f.aK(Unknown Source)
at com.groupdocs.conversion.operations.F.aJ(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.aI(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)
Even by increasing the limit it stills giving the same error.

So my question is Can we use “GroupDocs.Conversion” in an osgi environment ?

Thanks for helping,
Eric

2

@enrico44,

Thanks for taking interest in GroupDocs.Conversion for Java and posting your concerns.

GroupDocs.Conversion for Java is a back-end API that could be integrated in any Java application without platform dependency. The only requirement is Java run-time version 6 (1.6) and above. You can also see development environment details here.
For further investigation, can you please share following details:

  • Full exception stack trace
  • Complete development environment details
  • A sample project using that issue could be reproduced
3

Hi,

I just realize that the issue has nothing to do with OSGI, as I made another test, on the same linux server, same JDK, but pure simple java class :

package misc;

import java.io.File;

import com.groupdocs.conversion.config.ConversionConfig;
import com.groupdocs.conversion.handler.ConversionHandler;
import com.groupdocs.conversion.handler.ConvertedDocument;
import com.groupdocs.conversion.options.save.PdfSaveOptions;

public class Conversion
{

            public static void main(String[] args)
            {
                            File fileIn = null;
                            File fileOut = null;

                            if (args.length > 0)
                            {
                                            fileIn = new File(args[0]);
                                            fileOut = new File(args[0] + ".pdf");
                            }
                            else
                            {
                                            System.exit(-1);
                            }

                            System.out.println("converting " + fileIn.getPath());
                            // TODO Auto-generated method stub
                            ConversionConfig config = new ConversionConfig();
                            config.setOutputPath(fileOut.getParent().toString());
                            ConversionHandler conversionHandler = new ConversionHandler(config);
                            PdfSaveOptions pdfSaveOptions = new PdfSaveOptions();

                            ConvertedDocument result = null;
                            try
                            {
                                            result = conversionHandler.convert(fileIn.getPath(), pdfSaveOptions);
                            }
                            catch (Exception e)
                            {
                                            // TODO Auto-generated catch block
                                            e.printStackTrace();
                            }
                            result.save(fileOut.getName());
            }

}

It gives the same exception :

[cagertt@gert-tst@/APPL/GERT/appl/fuse/tools/misc]$ ./Conversion.sh /APPL/GERT/appl/fuse/tools/misc/test.eml
converting /APPL/GERT/appl/fuse/tools/misc/test.eml
Exception in thread “main” java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at com.aspose.words.internal.aM.j(Unknown Source)
at com.aspose.words.afZ.handleText(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xa.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xe.read(Unknown Source)
at com.aspose.words.aa.b(Unknown Source)
at com.aspose.words.aa.a(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.groupdocs.conversion.domain.savers.email.e.c(Unknown Source)
at com.groupdocs.conversion.domain.documents.f.a(Unknown Source)
at com.groupdocs.conversion.operations.f.aK(Unknown Source)
at com.groupdocs.conversion.operations.F.aJ(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.aI(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)

    at misc.Conversion.main(Conversion.java:38)

Conversion.sh :

java -cp .:$FUSE/karaf/data//repository/com/groupdocs/groupdocs-conversion/18.12/groupdocs-conversion-18.12.jar misc/Conversion $1

The JDK is :

[cagertt@gert-tst@/APPL/GERT/appl/fuse/karaf]$ java -version
openjdk version “1.8.0_171”
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)

Linux version
Operating System: Red Hat Enterprise Linux Server 7.3 (Maipo)
CPE OS Name: cpe:/o:redhat:enterprise_linux:7.3:GA:server
Kernel: Linux 3.10.0-514.26.2.el7.x86_64
Architecture: x86-64

So the issue is more related to JDK itself, or due to the lack of a valid licence ???

I can just insist to look at this, as we are really pleased by the result and we will probably buy your product.

Best regards,
Eric

4

@enrico44,

Thanks for sharing more information, it’ll surely help us in further investigation.

Please see API development environment, installation and configuration details [here]( How to run examples | Documentation). GroupDocs.Conversion for Java supports Java run-time version 6 (1.6) and above.

Are you evaluating the API in trial mode (without applying a valid license)?
There are certain limitations when using the API in trial mode. However, you can post a request for a temporary license which is a time-restricted full license that lets you test every aspect of the API. To post request for temporary license, please go here. Follow the wizard and on step 5 you can request a temporary license. This document can properly guide you - Wizard.zip (152.9 KB)

Can you please tell us how did you increase the limit? What opts you set to increase the limit?
Also, please share the problematic file with us.

5

As we run on Linux (Red Hat Enterprise Linux Server 7.3), with JDK 1.8 (openjdk version “1.8.0_171”), I think we are good.

I just received from you a temporary license, I use it now.

I increase the limit like this, I think that 8Gig is enough :

[cagertt@gert-tst@/APPL/GERT/appl/fuse/tools]$ java -Xms4G -Xmx8G -cp .:$FUSE/karaf/data//repository/com/groupdocs/groupdocs-conversion/18.12/groupdocs-conversion-18.12.jar misc/Conversion /APPL/GERT/appl/fuse/tools/test.eml
licence valid
converting /APPL/GERT/appl/fuse/tools/test.eml
Exception in thread “main” java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at com.aspose.words.internal.aM.j(Unknown Source)
at com.aspose.words.afZ.handleText(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xa.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xe.read(Unknown Source)
at com.aspose.words.aa.b(Unknown Source)
at com.aspose.words.aa.a(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.groupdocs.conversion.domain.savers.email.e.c(Unknown Source)
at com.groupdocs.conversion.domain.documents.f.a(Unknown Source)
at com.groupdocs.conversion.operations.f.aK(Unknown Source)
at com.groupdocs.conversion.operations.F.aJ(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.aI(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)
at misc.Conversion.main(Conversion.java:56)

I attach data_src.zip (5.9 KB)
the source and the eml file

6

@enrico44,

It seems more like a environment/server issue. Application is not getting required/sufficient memory to execute the process. You still have to provide sufficient memory to the application/process.
As your server VM is 64-Bit. If your heap size is less than 32 GB, please use -XX:+UseCompressedOops in your opts. It will stretch memory. Let us know if it helps.

7

I tried with 32Gb, also with -XX:+UseCompressedOops, same wrong result.

I didn’t manage to get a stack trace with lines info on each line (I tried with a JDK in place of JRE), can you tell how it is possible ?

Also I send to you the heapdump (you will see that the consummed memory is very low when exception occurs)

[cagertt@gert-tst@/APPL/GERT/appl/fuse/tools]$ java -XX:+HeapDumpOnOutOfMemoryError -cp .:$FUSE/karaf/data//repository/com/groupdocs/groupdocs-conversion/18.12/groupdocs-conversion-18.12.jar misc/Conversion /APPL/GERT/appl/fuse/tools/test.eml
converting /APPL/GERT/appl/fuse/tools/test.eml
java.lang.OutOfMemoryError: Requested array size exceeds VM limit
Dumping heap to java_pid45179.hprof …
Heap dump file created [188994778 bytes in 0.671 secs]
Exception in thread “main” java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at com.aspose.words.internal.aM.j(Unknown Source)
at com.aspose.words.afZ.handleText(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xa.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xe.read(Unknown Source)
at com.aspose.words.aa.b(Unknown Source)
at com.aspose.words.aa.a(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.groupdocs.conversion.domain.savers.email.e.c(Unknown Source)
at com.groupdocs.conversion.domain.documents.f.a(Unknown Source)
at com.groupdocs.conversion.operations.f.aK(Unknown Source)
at com.groupdocs.conversion.operations.F.aJ(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.aI(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)
at misc.Conversion.main(Conversion.java:56)

8

@enrico44,

As you are working in a VM environment. Can you please share hardware resources that you’ve dedicated to the VM:

  • How much RAM size/limit is siphoned from your system to VM? Did you try to increase siphoned RAM size (This number can be changed easily under the VM settings menu)?
  • Hard drive size

It is clear from your exception that application crashes because it is trying to allocate an array larger than the JVM can support.
You can also reproduce this exception if you run following line of code:

long[] test = new long[Integer.MAX_VALUE];

Please provoke Integer.MAX_VALUE for a test.

Also, please share a sample project (along-with steps to compile it) with us using that issue could be reproduced.

9

Indeed, we work with VM Linux. It has 64G memory, more than enough … 200 Gb free disk.
Your test (new long) gives this :
java -Xmx32G misc/ExceptionOO
will do it for size 2147483647
Exception in thread “main” java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at misc.ExceptionOO.main(ExceptionOO.java:10)

I forward you a maven project with the sample code

To test outside eclipse, at the prompt :

java -cp pdf_test-0.0.1-SNAPSHOT.jar:$FUSE/karaf/data//repository/com/groupdocs/groupdocs-conversion/18.12/groupdocs-conversion-18.12.jar com/mycompany/pdf/App /APPL/GERT/appl/fuse/tools/GroupDocs.Conversion.lic /APPL/GERT/appl/fuse/tools/test.eml

the first jar is the jar build by maven under target folder, the second your jar. Parameters are the path of licence file, and the test eml file

pdf_test.zip (490.3 KB)

10

@enrico44,

Thanks for sharing the details.
Can you please confirm, the project you shared has a output file with evaluation tag. Did you get this output using the same project on your server?
Are you able to apply license properly?

11

The evaluation tag is not there, as the temp license file is used and thus works.
I used the same jar file of the project on the server. I even tried on a second VM server (less busy) and it gives the same error.

It seems indeed related to the fact that it is running on VM rather than a physical machine. Do you have VM at your company to test it ?

12

@enrico44,

The project you shared has a output file with evaluation tag - file.eml.zip (116.5 KB)
Aside from this exception/error, do you still get output file on VM (as the project you shared has a output file)?
We are investigating this scenario at our end. Your investigation ticket ID is CONVERSIONJAVA-565. As we have any further updates on it or if we require further information, we’ll notify you.

13

You’re wright sorry, I made so many tests … I forgot to apply the license file in the code.

Here is the good version
pdf_test.zip (541.1 KB)

on the server, it stills goes in error, it gives this :

[cagertu@gert-uat-s02@/APPL/GERT/appl/fuse/tools]$ java -cp pdf_test-0.0.1-SNAPSHOT.jar:$FUSE/karaf/data//repository/com/groupdocs/groupdocs-conversion/18.12/groupdocs-conversion-18.12.jar com/mycompany/pdf/App /APPL/GERT/appl/fuse/tools/GroupDocs.Conversion.lic /APPL/GERT/appl/fuse/tools/test.eml
license file is used and lic.isValidLicense()=true
converting file /APPL/GERT/appl/fuse/tools/test.eml
conversionHandler.isLicensed() true
Exception in thread “main” java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at com.aspose.words.internal.aM.j(Unknown Source)
at com.aspose.words.afZ.handleText(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.o(Unknown Source)
at com.aspose.words.afZ.n(Unknown Source)
at com.aspose.words.afZ.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xa.a(Unknown Source)
at com.aspose.words.Xa.c(Unknown Source)
at com.aspose.words.Xe.read(Unknown Source)
at com.aspose.words.aa.b(Unknown Source)
at com.aspose.words.aa.a(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.aspose.words.aa.(Unknown Source)
at com.groupdocs.conversion.domain.savers.email.e.c(Unknown Source)
at com.groupdocs.conversion.domain.documents.f.a(Unknown Source)
at com.groupdocs.conversion.operations.f.aK(Unknown Source)
at com.groupdocs.conversion.operations.F.aJ(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.converter.pdf.h.a(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.aI(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)
at com.groupdocs.conversion.handler.ConversionHandler.convert(Unknown Source)
at com.mycompany.pdf.App.main(App.java:75)

14

To make a last statement, I made the same test but on a old IBM AIX server that we no more use, and it is working there : test.eml.pdf (33.4 KB)

So the VM track seems the good one

You can also see that there is another issue, the “Sent” field of the mail is totally wrong, it should be a date/time. Is this issue known ?

15

@enrico44,

Can you please confirm, do you get output PDF as well along-with exception? Because we assume that the project you shared with us is processed on VM.

This server is on Vm?

16

No, the result you get in the project is obtained on my eclipse dev environment, not on the VM. On the VM Linux, it’s going allways wrong.

As far as I know, the AIX server is not a pure VM like on the Linux VM, it’s more than 10 year old.

17

@enrico44,

We are also investigating this issue. Your investigation ticket ID is CONVERSIONJAVA-566. As we have any further update, we’ll notify you.

This is also under investigation.
We’ll appreciate your patience in this regard.

18

Hi, a colleague told me check ulimit values, maybe that it can help :

[cagertt@gert-tst@~]$ ulimit
unlimited
[cagertt@gert-tst@~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 256953
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 4096
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

19

Hi, the same colleague found this :

It seems that under openjdk we cannot allocate the size MAX_VALUE , but it is MAX_VALUE - 3 :
I made the smal pgm class misc.ExceptionOO with the line
long[] test = new long[Integer.MAX_VALUE - x];

[cagertt@gert-tst@/APPL/GERT/appl/fuse/tools]$ java -Xmx32G misc.ExceptionOO 0
array size will be 2147483647
Exception in thread “main” java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at misc.ExceptionOO.main(ExceptionOO.java:11)
[cagertt@gert-tst@/APPL/GERT/appl/fuse/tools]$ java -Xmx32G misc.ExceptionOO 1
array size will be 2147483646
Exception in thread “main” java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at misc.ExceptionOO.main(ExceptionOO.java:11)
[cagertt@gert-tst@/APPL/GERT/appl/fuse/tools]$ java -Xmx32G misc.ExceptionOO 2
array size will be 2147483645
Exception in thread “main” java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at misc.ExceptionOO.main(ExceptionOO.java:11)
[cagertt@gert-tst@/APPL/GERT/appl/fuse/tools]$ java -Xmx32G misc.ExceptionOO 3
array size will be 2147483644
done 2147483644

So the max size is Integer.MAX_VALUE - 3

20

@enrico44,

Thanks for the details.
We are still investigating the ticket CONVERSIONJAVA-565, we’ll update you as we have any update.