6 Replies Latest reply on Aug 13, 2010 11:19 AM by jfclere

    Unicode convert problem with Jboss in Linux

    isurux

      Hello,

       

      I have faced with an issue where UTF-8 characters are not correctly formatted ONLY in the Jboss (jboss-5.1.0.GA-3) instance running on Linux.

       

      For an instance: BORÅS is converted to BOR?S at Linux jboss instance.

       

      When I copy and configure the same jboss instance to run at Windows (SP3) it works perfectly.

       

      Also I have change the default setting in Linux by including JAVA_OPTS=-Dfile.encoding=UTF-8 in .bashrc and run.sh files.

       

      Any suggestions or workarounds ?

        • 1. Re: Unicode convert problem with Jboss in Linux
          jfclere

          Where does the problem happend?

          When serving files (add an example)?

          When in a servlet/application/database?

          • 2. Re: Unicode convert problem with Jboss in Linux
            isurux

            Hello Clere,

             

            Thank you for the prompt reply.
            I have exposed a jsp to upload a xml file using the post method. I will be obtaining the file as a FormFile using struts in the Action class as follwos:
            FormFile file = theForm.getFile();
            Secondly, I retrieve the file data as byte array
                    byte[] buf = file.getFileData();
            When I view the result of above buf variable at runtime or if I write it to a file using FileOutputStream it is clear that the unicode has been converted to a non recognized character.
            • 3. Re: Unicode convert problem with Jboss in Linux
              jfclere

              JAVA_OPTS=-Dfile.encoding=UTF-8

              Is probably the way to go. But you should try to check the encoding in windows.

               

              public class Test {
                public static void main(String[] args) {
                System.out.println(sun.io.Converters.getDefaultEncodingName());
                System.out.println(System.getProperty("file.encoding"));
                }
              }
              

               

              Check the output in windows and in Linux. Recheck with -Dfile.encoding=windows_value on Linux.

              • 4. Re: Unicode convert problem with Jboss in Linux
                isurux

                Hello Clere,

                 

                I have tired out that by passing the '-Dfile.encoding=UTF-8' to the run.sh file and check the file.encoding at runtime but same result.

                 

                Now I have identified the problem is at the server side and not in the view.

                 

                I tried to save the byte array obtain from the File into an xml file in jboss tmp folder and it contains the correct Unicode.

                 

                In my scenario the uploaded file is straight away sent to a message queue to be processed by an MDB. So the flow is as follow:

                 

                Upload the XML ----> Obtain the byte[] from the file ---> put it on a JMS queue ---> picked by a MDB to process

                 

                When the posted Message is cast to a TextMessage and process, the Unicode is been incorrectly converted.

                 

                Is there is any specific configuration I have to do in the JMS queue in order to convert it to the correct Unicode?

                • 5. Re: Unicode convert problem with Jboss in Linux
                  isurux

                  Finally I was able to find a solution, BUT the solution is still a blackbox. If anyone have the answer to WHY it has failed/successful please update the thread. Till it is solved I will let the thread open.

                   

                  Solution at a glance :
                  1. Captured the file contents as a byte arry and wrote it to a xml file in jboss tmp folder using FileOutputStream
                  2. When posting to the jboss Message queue, I used the explicitly wrote xml file  (1st step) using a FileInputStream as a byte array and pass it as the Message body.
                  Code example:

                  View: JSP page with a FormFile
                  Controller Class :UploadAction.java
                  public ActionForward execute(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response){
                     ...........
                    
                     writeInitFile(theForm.getFile().getFileData()); // Obtain the uploaded file
                     Message msg = messageHelper.createMessage( readInitFile() ); // messageHelper is a customized factory method to create Message objects. Passing the newly   
                     wrote file's byte array.
                     messageHelper.sendMsg(msg); // posting in the queue
                     ...........
                  }
                  private void writeInitFile(byte[] fileData) throws Exception{
                    
                     File someFile = new File("/jboss-5.1.0.GA-3/test/server/default/tmp/UploadTmp.xml");  // Write the uploaded file into a temporary file in jboss/tmp folder
                     FileOutputStream fos = new FileOutputStream(someFile);
                     fos.write( fileData );
                     fos.flush();
                     fos.close();   
                  }
                  private byte[]  readInitFile() throws Exception{
                     StringBuilder buyteArray=new StringBuilder();
                     File someFile = new File("/jboss-5.1.0.GA-3/test/server/default/tmp/UploadTmp.xml");      // Read the Newly created file in jboss/tmp folder
                     FileInputStream fstream = new FileInputStream(someFile);
                     int ch;
                     while( (ch = fstream.read()) != -1){
                               buyteArray.append((char)ch);
                     }
                     fstream.close();
                     return buyteArray.toString().getBytes();   // return the byte []
                  }
                  Foot Note: I think it is something to do with the Linux/Windows default file saving type. eg: Windows default : ANSI.
                  • 6. Re: Unicode convert problem with Jboss in Linux
                    jfclere

                    return  buyteArray.toString().getBytes();   // return the byte []

                    This is going to convert the String into a a Byte[] using the platform encoding... that is why it works.