Spring Hibernate Accounting App

At the beginning of December 2015, I was starting to work on this Malaysia-based client project. A web based accounting app.

This app is really huge, containing more than 20 accounting modules, such as Budgeting, General Ledger, Account Payables, Account Receivables, Loan, Staff Loan, Payroll, etc. We have at least 30 programmers and 20 system analysts to work on this project.

It's time to put my Intellij IDEA to a good use.

When I joined, this version of the app has been developed for about 6 months. While those modules are being actively developed, the framework was being migrated from framework-less servlet based web app (yes, naked servlet in 2015!) to a Spring-JDBC based web app.

To add more stress to the developers, the Spring-JDBC version is also being migrated to Spring-Hibernate. Fortunately, we have a good consultant team (thanks Sholleh and his crew!) guiding us in this migration process.

With the belief that Spring-Hibernate version will be much more maintainable than its Spring-JDBC and naked servlet counterparts, we developers are forced very happy to do the conversion.

In addition to the naked servlet case as mentioned above, we are still using SVN as source control. Come on! We are in the everything-distributed era. Distributed team. Distributed source control. Please use Git or Mercurial on the next projects.

We are using Oracle 12c as the database. By the time I joined, developers were using shared database for developing and testing. QA and SA tests were using the same database. The Jenkins continuous integration was also using the same database. Even client tests were using the same database. We never really were sure where an error came from. Did a database locking come from a bug created by me or other developers? We couldn't be sure. Who delete my test data? We didn't know unless the perpetrator confessed. I was the first developer to install and use a local database for development and invited other developers to do same. Unfortunately, without enforcement and hardware (memory chip) support from the company, most developers won't install an Oracle 12c DBMS in his PC/laptop.

It is pretty simple to install local oracle database if you are working with Windows or Linux computers. If you are using Mac computer like me, you have to make an additional effort. The simple options are to use a virtual machine or docker. I chose to use docker. Be just executing this one-liner, you got an Oracle DMBS running on your Mac:

docker run -d -p 8080:8080 -p 1521:1521 -v oracle_data:/u01/app/oracle sath89/oracle-12c

If you are still using older version of Mac OS X that requires docker-machine, use this docker command instead:

docker run --shm-size=1024MB -d -p 8080:8080 -p 1521:1521 -v oracle_data:/u01/app/oracle sath89/oracle-12c

At the beginning of migration from JDBC to Hibernate, some developers got the hibernate proxy error. In the process of solving this problem, they resorted to changing all JPA entity columns to eager-fetch. Ouch. There is a reason that lazy-fetch is set as the default. It is to avoid loading the whole world to the memory. The solution to the hibernate-proxy-error is really just to move database interactions from @Controller methods to @Transactional @Service methods. Keep everything lazy-fetched by default. If you want something to be eager-fetched, do it using JPQL/HQL.

A common error we got was the jackson-serialization-error. If you tried to find the error at Services code, you will never find it. Serialization only happens when Java is about to send a Java object to an external medium. In this case, a controller was about to send a Java object to a web browser over HTTP response. When there is a reference loop in the Java object, the serialization will never end, causing the jackson-serialization-error. The solution for this error is pretty simple. In the Controller method returning the Java object, you just need to nullify one or more Java object properties causing the reference loop, and you are done.

By the time I started working, there was no action authorization feature. A consultant team has been assigned to work on this feature, but they require more time. We plan to use JWT for the authorization. I didn't know anything about JWT yet. The client wanted to see some authorization features while testing my module in the next few days. So I added this annotation:

@Retention(RetentionPolicy.RUNTIME) public @interface UrlAccessControl {     String[] perihalPeringkat() default {"PTJ", "JAB", "BN", "PKN"};  // User rank     String[] perihalPeranan() default {"Penyedia", "Penyemak", "Pelulus"};  // User role } 

Added a controller interceptor:

@Component public class SecuredControllerInterceptor implements HandlerInterceptor {      @Override     public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {         if (handler instanceof HandlerMethod) {             HandlerMethod handlerMethod = (HandlerMethod) handler;              UrlAccessControl urlAccessControl = handlerMethod.getMethodAnnotation(UrlAccessControl.class);             if (urlAccessControl != null) {                ...             }         }         return true;     } } 

Configured the interceptor:

@Configuration @EnableWebMvc @ComponentScan("com.thetaedge") public class OneSpeksWebConfig extends WebMvcConfigurerAdapter {      @Autowired     SecuredControllerInterceptor securedControllerInterceptor;      @Override     public void addInterceptors(InterceptorRegistry registry) {         registry.addInterceptor(securedControllerInterceptor);     } }

And decorated some controllers with the new annotation:

@UrlAccessControl(perihalPeringkat = {"JAB", "PKN"}, perihalPeranan = {"Penyemak", "Pelulus"}) @RequestMapping(path = "/semak", method = RequestMethod.POST, params = {"peringkat", "jenisAnggaran", "grid"}) @ResponseBody public void semak( HttpSession session, @RequestParam(value = "jenisAnggaran") Long kodJenisAnggaranPkid, @RequestParam(value = "grid") String gridStr ) throws IOException { ... }

That worked for the coming test :D

I also found out that developers always returning success status 200 from ajax calls, even when the ajax response was expected to trigger an error message pop up in the UI. And they were returning error response using a normal return statement in the Controller. Even when the error happened far in a block in a service, they still managed to manually return control to the Controller and made the Controller returning an error object. We shouldn't do that. Errors are not a normal thing, so returning error should be done in a dramatic way by throwing a RuntimeException. But, how would you show a JavaScript error message popup in the browser if you throw a Java Exception in the backend? Let me show you how to do it.

First, you define a RuntimeException class that you want to throw when there were errors:

public class AmountExceedsTheSpecifiedLimitException extends RuntimeException {     public AmountExceedsTheSpecifiedLimitException() {         super("Amaun melebihi batas");     } } 

Second, you define a controller method to handle runtime exceptions:

@ExceptionHandler(RuntimeException.class) @ResponseBody public Map jsonErrorResponse(Exception ex, HttpServletResponse response) { Map errorMap = new HashMap<>(); errorMap.put("errorMessage", ex.getMessage()); StringWriter sw = new StringWriter(); PrintWriter pw = new PrintWriter(sw); ex.printStackTrace(pw); String stackTrace = sw.toString(); System.out.println(stackTrace); errorMap.put("stackTrace", stackTrace); response.setStatus(HttpStatus.INTERNAL_SERVER_ERROR.value()); return errorMap; }

Third, throw the exception in your error checking code in one of your Spring service (or anywhere else):

                if (votAmountThn1After.compareTo(batasMengurusJabatan.getBatasLulust1()) > 0) {                     throw new AmountExceedsTheSpecifiedLimitException();                 }

Fourth, add an error callback to your ajax call, and add the code to show error pop up in that error callback:

  $.ajax('... the URL ...', {    success: function (perihal) {                             ....    },    error: function (xhr, error, standardErrorTitle) {     if (xhr && xhr.responseJSON && xhr.responseJSON.errorMessage) {      var errorMessage = xhr.responseJSON.errorMessage;      $.messager.alert('Notification', errorMessage, 'error');     }    }   });

Those are the four steps to connect your JavaScript message box to your Java RuntimeException.

We don't have specialized programmers to write front-end/JavaScript code. The Java programmers are also expected to write JavaScript. Most of them wrote JavaScript directly inside JSP template files. After some periods, they willingly move the JavaScript from template files to its dedicated JavaScript files. Because of lacking JavaScript knowledge, they wrote everything in the global namespace. I made some examples of avoiding global namespace pollution by putting my JavaScript code inside a closure and exposing global names via JS object based namespace. Instead of writing all global functions like this code:

function functionA() {  }  function functionB() {  } 

I wrote this code instead:

var Namespace1 = (function ($) {  function functionA() {   }   function functionB() {   }   // Make some functions/names public, and merge the names to the existing Namespace1 namespace  return $.extend(true, Namespace1, {   Namespace2: {    Namespace3: {     globalFunctionA: functionA    }   }  }); })(jQuery);

This way, you can choose which functions to expose globally, which functions you want to keep private. You will only put one name in the global namespace, in this case, the Namespace1 object. But without support from the management who schedule project activities, nobody will update their code to follow this practice.

We are using a quite decent jQuery based UI library called jQuery EasyUI. It has data grids, combo grids, input box, and dialog library that we use a lot. Developers in this project found an interesting use of datagrid column formatter:

function getDaerah(pkid) {  if (!pkid) {   return '';  }  var response = $.ajax({      url: contextPath + '/spring/combo/getDaerahByPkid/' + pkid,      dataType: 'text',      async: false  });  return response.responseText; }

So initially, the table data loader will only request IDs from the backend. When the web browser is about to display the data, the loader will again make a backend request to load appropriate human readable description representing the ID. The problem with the code above is that it uses

async: false

. This will switch ajax into synchronous mode. The problem with synchronous ajax is that it will halt all JS execution in order to wait for the ajax request result. This will be a bad user experience. Browser creators tried to warn developers about this problem by showing this warning in the developer console.

We really have to avoid synchronous ajax. In order to do that, change the formatter code above to

function getDaerahWrapper($grid) {  var cache = {};  return function(pkid, row, index) {   if (!pkid) {    return '';   }   var cachedVal = cache[pkid];   if (cachedVal) {    return cachedVal;   }    var response = $.ajax({    url: contextPath + '/spring/combo/getDaerahByPkid/' + pkid,    dataType: 'text',    async: true,    success: function (text) {     cache[pkid] = text;     $grid.datagrid('refreshRow', index);    }   });   return 'Loading...';  }; }

After that, we need to change formatter setting from

formatter: getDaerah

to

formatter: getDaerahWrapper($grid)

This change gives us two benefits. First, we eliminated synchronous ajax. Second, we used caching, so the ajax request using a specific pkid will only be done once.

It is sad that I am the only one writing automated tests in this project. Project this huge... I am trying to write unit tests, integration tests, and selenium (functional) tests for the code I wrote, hoping others someday will do the same, and hoping someday management will realise the importance of automated testing. Because actually, automated tests require full support from management, so they include it as a part of project scheduling.

Setting Selenium testing, integration, and unit testing is pretty simple.

You need to add these dependencies to the build.gradle:

dependencies{     testCompile "org.springframework:spring-test:4.+"     testCompile group: 'org.seleniumhq.selenium', name: 'selenium-java', version: '2.+'     testCompile group: 'junit', name: 'junit', version: '4.+'     testCompile "org.mockito:mockito-core:2.+" } 

For the Selenium tests, you need to implement the Page Object pattern. In the Page Object pattern, you must wrap all calls to Selenium Web Driver API functions inside Page Objects. All functional test functions must not contain any direct call to the Selenium Web Driver API. This way, the functional test will be much more maintainable.

In both functional and integration tests, you want to see real interactions with the database. Therefore, you need to inject real service objects using @Autowired. To enable this behaviour, you need to add these annotations to the functional and integration test classes:

@RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration(classes = {OneSpeksWebConfig.class}) @WebAppConfiguration 

OneSpeksWebConfig is the Spring WebMVC configuration class that you would implement like this:

@Configuration @EnableWebMvc @ComponentScan("com.thetaedge") public class OneSpeksWebConfig extends WebMvcConfigurerAdapter {    ... }

In unit tests, sometimes you need to mock some classes. There are reasons for doing this mocking. For example, you don't want your code to trigger database access or network connection. However, you need to be careful when mocking classes. You must avoid mocking classes you don't own. You need to create a wrapper to that third party class and mock that wrapper. You can create integration tests of that wrapper afterwards. Your unit test will access the mock of that wrapper. Mockito is my favorite library to create mocks.

Trying to be accountable for the use of my time in this project, I am using the Time Doctor app. I haven't used all of its features but someday I really want my client to use its payroll feature, so my income from a particular client would also be recorded in Time Doctor.