Performance Improvement in Oracle Form Loading with UIRnR Record&Replay

This article will explain how we have improved the form loading time.  We have found the following areas where performance can be improved.

• • Java Extension Library for UIRnR agent

  • Javascript Injection into web pages

  • No.of threads and their execution

  • UIRnR agent logs

  • Socket Creation

  • HttpURLConnection

Java Extension Library for UIRnR agent

Java agent is for transforming the classes while JVM is loading them. UIRnR uses the agent to add listeners to each component. Once the listeners added, UIRnR can record and replay Java Swing/AWT components. Any class that agent uses should be loaded prior to agent invoking. This can be done by loading the required jars either by using bootstrap class loader or Java’s extension class loader.

UIRnR uses java extension mechanism to load the required classes as shown below.

-Djava.ext.dirs = Path of UIRnR Client installation

Example : -Djava.ext.dirs = C:\Users\pneeli\tre\UITester\hletkdevglobaltriniticom

We checked that agent for UIRnR agent requires only 6 jars to work. So, we moved them to a separate folder and specified that as the java extension directory. With this change, Oracle forms are loading fast.

To understand the actual problem, we have taken the form loading time for the below cases.

We have observed that form loading time is increasing as the no of folders increases in the specified extension directory.

Javascript Injection into web pages

UIRnR will inject few javascript files into web pages during record and replay. This helps UIRnR in getting calls from injected javascript code to record and replay actions. If there is nothing to record, no need for the injection.

In any Oracle EBS, when we click on any oracle form, it will launch a new browser window which contains only an embedded applet code. This page is actually responsible for opening the oracle form. We are injecting JS code into this page also as the part of recording and replaying.

After stopping injection into that browser window, we have seen the improvement ( ~ 10 seconds ) in Form loading.

No.of threads and their execution

Using Java Mission Control, we have found that the following two threads are using high CPU usage and java heap space. These two threads will be started from the java agent.

Source Label configuration thread - Source Label is an oracle form GUI component’s property which is unique. To get that property, we need to configure one field value in an Oracle class (oracle.forms.Main), using Java reflection. It is a one-time activity during the initialization of the oracle form. As per existing logic, execution is not coming out of while loop even though source label configuration is done.

Now we are stopping the thread once the configuration is done (with a code level fix).

Listeners Adding thread -  This thread is used to add event listeners to each Oracle component to handle the actions during recording and replaying. And this should be running continuously as new components might be adding on the go when you do actions further in the idle form. This thread has also a while loop and there is no delay for each iteration. An infinite loop without any delay will consume CPU cycles.

After adding 10 milliseconds delay, we have observed less CPU usage and memory usage allocated for it.

UIRnR agent logs

We have configured Java Agent in JAVA_TOOL_OPTIONS as browser session level environment variable. The agent will start running once the JVM initialization starts. For the agent logging purpose, we have used log4j to write logs to a file. In our existing configuration, we have the following property.

log4j.appender.file.MaxFileSize = 100MB

Above line in log4j configuration properties, decreases application performance as it uses some CPU cycles and other resources (memory, etc) to write the logs into a huge 100MB file.

After writing logs in small chunk files(3MB) with the help of log4j file rotation mechanism, we have seen performance improvement in Oracle Form loading.

In total, form loading delay reduced around 25 sec, in our local instance. When checked in client instances, delay reduced around 3 min.

We have used the following EBS instance for all the above activity.

http://hlr12dev.global.triniti.com:8012

Time values shown in this article are taken for the following form and may vary from form to form.

Application Developer->Profile

Socket Creation

We have noticed some performance issue with one of the client instances. But this time, we have observed that there is a performance issue with HTML web page loading also along with form loading. The following are the loading times (in sec) taken on that instance.

Note: We have taken the web page loading time using IE’s Network Profiler from where we can capture the response timings for each request the browser makes to the server.

To find out the root cause, we have gone through different levels. First, we have tried removing the UIRnR agent configuration and haven’t seen any major time difference. Next, we have tried removing the javascript injection too, but still no time difference.

If it is not java agent and javascript injection, then the remaining check left is proxying the requests and responses.  The typical web proxying concept depicted below.

Figure 1: Web Server Bypassing

Above proxying concept is well explained theoretically in Web techniques: Your Own Proxy Server

We have noted the timings for gathering each request from the client, sending it to the server, modifying the response taken from the server and forwarding it back to the client (browser). None of them show any much difference in time.

We also checked the socket creation and closing logic. We are maintaining a pool of 20 client local ports in a linked list, then creating a socket by removing one port from the list and adding it to list when we close the socket ( for reusing it ).

When a page load makes 15 requests, then 15 ports from the list will be taken.

When a page load makes 20 requests, then all 20 ports will be used.

When a page load makes 25 requests, then 20 ports will be used and to make remaining requests, it has to wait for any existing request completion as the port will be added to the list, so that the 21st request will use that port to bind the host address. But we are getting the following exception whenever it tries to connect with the freed local port.

        java.net.bindexception: Address already in use connect

It is clear that the address it is trying to connect is being used by another process. When we check, we come know that there is a possibility that port can not be used for some time even after closing socket. FYI, Socket.setReuseAddress.

We have found that there is an exact delay of 21 sec from the following call to catching the exception.

        socket.connect(new InetSocketAddress(port));

Even though each request making runs on a different thread,  due to this exception, showing the HTML content on the browser is taking time.

As per Javadoc, a local port of Zero will let the system pick up a free port in the bind operation.

When we pass 0 as local port while binding address to the socket, as shown below

        socket.connect(new InetSocketAddress(0));

we didn’t get any exceptions ( saving 21sec ) and performance got improved.

Now with the above fix,

The time difference between the case (i) and (iii) can be ignored as bypassing the web server through a proxy server will consume some time than usual direct HTTP.

Time values shown in Socket Creation part of this article are taken for the following form and may vary from form to form.

Application Developer->Profile

HttpURLConnection

Even after changing from fixed no. of ports to the dynamic port allocation for each request while creating the socket, we have observed that form loading is taking so much time for the following form on DEMO.TRINITI.COM.

Order Management Super User, Vision Operations (USA) -> Orders, Returns -> Sales Orders

Using Fiddler, we have found that, the above form is making around ~500 requests to the web server and the turnaround time for each request is about 600-800ms which is unusual when compared with the time taken without UIRnR which is 200-300ms. Found that the first call to the Socket's InputStream.read() method(A blocking I/O call used to read the bytes one by one) is causing that ~400ms extra delay. We tried reading data with buffered I/O streams and Java's Non-Blocking implementation, but still no difference in performance.

HttpURLConnection is another way of establishing the connection to the web server to send an HTTP request and to get its response back. It is a URLConnection implementation with support for HTTP-specific features. Now, UIRnR will use HttpURLConnection to make requests from HTTP sites and with this, performance is improved a lot.

Without UIRnR, above metioned form is loading in 1m 45s. Following statistics are taken on DEMO instance with UIRnR.

We have also taken statistics for other EBS instances and consolidated them in Statistics on Oracle Form loading

Here, the time taken to load the form in the recording mode is equal to the time taken when launched without the tool.  Still, we can improve the performance in replay time. For that, we have analyzed the HTTP network traffic using the same Fiddler. Found that the no. of requests going to the server during replaying mode (~14000) are much higher than recording mode (~500) for the same form. After debugging, we come to know that a piece of javascript code is running for every 50ms when java lines are replaying and it is actually causing the delay. After increasing the interval delay from 50ms to 2s, form loading time is reduced and become consistent and same during both record & replay.

We have also analyzed the CPU utilization with IE F12 UI Responsiveness tool. Following shows the difference between the load on CPU when scripting is in action.