Ant, JspC, and Other Horrors

Posted by Uncle Bob on 02/03/2007

I’ve been trying to precompile JSPs today. They reference custom tag libraries. What a joy…

You’d think it would be simple to translate JSP file into Java. Tomcat does it all the time, automatically.

I’d very much like to precompile my JSP files, because I want to write tests for them; and I don’t want to use Cactus, or have the server up for my tests.

I found the Jasper compiler, and the ant task that invokes it, but I thought it might be nice to run it from the command line just to see how it work. You know:

java org.apache.jasper.JspC myFile.jsp

You’d think that would be easy, wouldn’t you?

But NO. You have to include a zillion jars just to get the JspC compiler to run. Jars from apache.common, and appache.server, and commons.logging, etc. You even have to include the ant.jar file.

I consider that last to be completely sick. What the H___ does JspC have to do with Ant? Why in the begrosian devalent is there a dependency on ant, of all things??!!? Haven’t these people heard of Dependency Management?

sigh

Anyway, I gave up on the command line idea because apparently the Jasper compiler wants to see the web.xml file and so the whole web app has to be put together before you can run the compiler. I understand why they did this, but it sure is frustrating for someone who just wants to run the frickin compiler.

So I fell back on the ant task. Now I have to tell you that I hate ant. Ant was born during those sick sick days when people thought that XML was a cool grammar. XML is not a cool grammar. XML is a markup language. It works fine to encode data that machines can read, and humans can barely read, but it is by no means a natural syntax. Ant suffers from terrible keyword-itis, and the nasty inflexibility.

Inflexible you say? Why, can’t you write any ant-task you please?

Sure, so what. You think I want to read a bunch of java code to figure out how to invoke the JspC command line? Hell, all I want to do is compile one frickin JSP file into a JAVA file. If ant let me pass in command lines, like make or rake or some other reasonable build tool, I just might be able to do that.

But NO. The JspC ant task wants to compile the WHOLE web app for me. And it wants there to be a web.xml file that describes that web app.

sigh

So, like a good little ant slave, I put the ant build script for JspC into my build.xml file and fired it up. It took a little fiddling and cajoling. But eventually I got the JspC compiler to run. And what did it say?

/Users/unclebob/projects/Library/Library/build.xml:147: org.apache.jasper.JasperException: file:/Users/unclebob/projects/Library/Library/web/WEB-INF/pages/template.jsp(25,21) Unable to load tag handler class "com.objectmentor.library.web.framework.tags.ActionPathTag" for tag "library:actionPath"

(what is it about error messages nowadays that they have to be 253 characters in length? Why can’t we have nice little error messages?)

The problem is that the JSP file that I am trying to compile is using a custom tag, and apparently there is no way to get the JspC ant task to tell the JspC compiler the classpath of my tag handler. I’ve tried everything I could think of, and searched high and low on the net, but I can’t seem to figure out how to make this work.

So I’m done for the weekend. I was hoping to write a blog on testing JSPs this weekend while attending my son’s wrestling match (he’s likely to make it to state this year), but it’ll have to wait because there’s no internet at the match.

I guess I’ll play with Ruby instead.

UPDATE

While eating dinner, it hit me. If I set the CLASSPATH variable to include the jar with the tag in it, and if I invoke JspC from the command line (or with a java ant task) it might work. So I tried it, and…whaddyano? it worked just fine.

Here is the ant target I eventually used. Note the hideous dependencies required by JspC. You might find this target useful if you ever want to precompile jsps that use custom tags.

</fileset>

</fileset>

</fileset>

</classpath>

</java>

</target>

Now on to the wrestling match!

Not so fast.

Some of the java files generated from the JSPs don’t compile. There is some mismatch between the tag processing libraries or something. The generated code is hideous to read, and the problem is probably subtle. So I’m going to go to bed.

urg!.

WRESTLING and SUCCESS

So, here I am at the wrestling match. The thing about these matches is that I need to pay attention for about 6 minutes every two hours. That’s how often Justin wrestles. So, of course, I bring my laptop and sit on the bleachers working on code while the whistles are blowing and the parents are screeching and my butt slowly gets numb.

Justin won his first match. It was a slaughter. He’s strong and smart, and has a good chance to get into the state tournament.

I won my first match with JSP too. Apparently the compile environment of my IDE is not exactly the same as the compile environment that Jasper uses. But I was able to resolve that by simply setting the -compile switch on the JspC command line. This compiles the generate java files in place using what seems to be the correct compile environment. At least there are no compiler errors.

I also set the -mapped command line argument. This causes jasper to create a new Out(...) call for each line on the input jsp. This makes the java file a bit easier to read. Apparently this command line is set when tomcat automatically compiles the jsps, and so comparing my generated files and tomcats generated files is easier with this argument set.

I set up a simple unit test to see if I could create an instance of one of the generated servlets. It looks like this:

public class ManageJspTest extends TestCase {

public void testCreate() throws Exception {

HttpJspBase manage = new com.objectmentor.library.jsp.WEB_002dINF.pages.books.manage_jsp();

}

At first this didn’t compile because the servlet apparently uses the apache.commons.logging. stuff. So I had to put that in the classpath of the unit tests. Grumble! “Dependency management guys! Dependency Management. Why do my unit tests need to know about logging? sigh

Anyway: I can now compile a unit test that creates a servlet generated by Jasper!!!

This is very good news. I have my doubts about whether it will work however. The code generated by my ant script, and the code generated by tomcat are not exact matches. There is one segment that is markedly different. The following snippet is in the tomcat generated java file, but not in the ant-script generated java file.

static {

_jspx_dependants = new java.util.ArrayList(1);

_jspx_dependants.add("/WEB-INF/tld/LibraryTags.tld");

}

I don’t understand exactly why, but I think it must have something to do with a difference in the way the two environments view the tag libraries.

Anyway, I imagine that when I try to execute the servlets generated by the ant script, they will fail because this code is missing.

Anyway, I think I’ll go watch Justin wrestle some more.

Justin won his second match at Regionals. That put’s him in the finals for the tournament, and ensures that he has a spot in the Sectionals next week.

Justin doesn’t wrestle again until 4pm, so Ann Marie and I went home for a couple of hours. While there, I was able to set up a quick test to invoke the generated servlet. It looks like this:

public class ManageJspTest extends TestCase {

private MockPageContext pageContext;

private MockJspWriter jspWriter;

private JspFactory mockFactory;

public void testCreate() throws Exception {

jspWriter = new MockJspWriter(10000, false);

pageContext = new MockPageContext(jspWriter);

mockFactory = new JspFactory() {

public PageContext getPageContext(Servlet servlet, ServletRequest servletRequest, ServletResponse servletResponse, String string, boolean b, int i, boolean b1) {

return pageContext;

}

public void releasePageContext(PageContext pageContext) {

}

public JspEngineInfo getEngineInfo() {

return null;

}

};

JspFactory.setDefaultFactory(mockFactory);

HttpJspBase manage = new com.objectmentor.library.jsp.WEB_002dINF.pages.books.manage_jsp();

MockHttpServletRequest request = new MockHttpServletRequest();

MockHttpServletResponse response = new MockHttpServletResponse();

manage._jspService(request, response);

}

Note all the mocks! Here is where the tomcat guys did a nice bit of work. The JspFactory is used within the generated servlet to gain access to all the objects it uses. It turns out that you can override the default implementation of the JspFactory by simply calling setDefaultFactory. Nice! That’s Dependency Management!

I did a quick breakpoint at the very end of my test and, sure enough, there is output stored in the MockJspWriter! Unfortunately, it’s nowhere near enough. The output stops as soon as the first tag was invoked. Indeed, there is a wonderfully silent return right in the generated code. No exception, no error message, just silent failure. sigh. Here’s the generated code:

out.write("<form action=\"");

if (_jspx_meth_library_actionPath_0(_jspx_page_context))

return;

It was generated from this portion of the jsp.

<form action="<library:actionPath actionName="books/manage"/>" method="post" id="list_form">

The actionPath tag is one of our custom tags. All it does is much a controller path into an appropriate url format. (If you don’t understand that, don’t worry about it.) Clearly our custom tag is not being found. This is probably because the jasper compiler is not setting the _jspx_dependants variable properly. So now I need to figure out why…

For some reason, Jasper is not paying attention to the following line in my jsp file:

<%@ taglib uri="/WEB-INF/tld/LibraryTags.tld" prefix="library" %>

It’s not clear why, though I have to say that pattern of silent errors is really starting to bother me. An error message would certainly be nice.

Why is Jasper ignoring the taglib directive? The LibraryTags.tld file is there, and in the right place. The ActionPathTag.class file is in the classpath used to invoke jasper. I’ve played with the -uribase argument of the jasper compiler (for fun, read the description of this argument and see if YOU understand what it’s saying.) It’s a mystery…

Comments

Leave a response

- Brian Slesinsky about 16 hours later:
  - - Now you know why JSP’s are evil. There are large numbers of better template languages to choose from, so it’s no big loss.

- Dean Wampler about 1 hour later:
  - - Even James Duncan Davidson, the inventor of Ant, has publicly expressed his regrets that he chose XML as the format for ant files.

- Scott Carlson 1 day later:
  - - Hopefully Brian wasn’t refering to Tapestry as one of those templating languages. At least with JSPs there is some attempt at precompiles. In Tapestry, the magic between your HTML and your Java page are almost impossible to test outside of a full running application (though it can go in a Mock container).
    - Incredibly painful and sloooow.

- Ravi Venkataraman 2 days later:
  - - One reason why you’d need to test JSP pages in a non-GUI environment is if the JSP had a lot of Java code in it. Tag libraries are one insidious way such code gets into JSPs.
    - The way I look at JSPs is that they are a conduit to convey the information from the Web UI (Html) to the application server and vice versa.
    - All I need to know is what action is requested and what data was present in the web page. The server uses this data to generate the data for the next page. A parser interprets this and generates the appropriate HTML page and passes this to the next JSP page, which presents it.
    - I can test the parser separately, independent of the web server.
    - Since the main work gets done in the Java code (Java Beans and classes), I can test the business logic independent of the web UI.
    - That leaves me the simple task of validating that the UI is performing correctly, calling the correct Java Beans, etc. This is easy since I can easily modify the JSP pages on the fly with Tomcat running and see the changes.
    - As a result of this architecture, we never need to test the JSP pages outside the web server.
    - Come to think of it, why would one want to test the JSP page outside the web server? JSP pages must exist on the sweb server.

- Ravi Venkataraman 2 days later:
  - - David: “Why not automate it to make sure your changes to the business layer donâ€™t have unexpected effects on your display?”
    - Why should changes in business logic have any effects on my display? Isn’t that the whole idea of the Model-View-Controller paradigm – that the changes to the model (that are limited solely to the business rules) should not have an effect on the display? If the changes involve the UI, then we do have to test the UI anyway.
    - In any MVC implementation with a clear separation between Mpdel and View, the issue should never arise. The fact that this question arises is an indication that the design might need to be tweaked.
    - If the design is such that one must run the UI to test business rules that have no bearing on UI display, then, too, the design should be revisited.
    - I simply do not see what additional benefit would be gained by testing JSP pages outside the container compared to testing the business rules alone.

- Uncle Bob 3 days later:
  - - JSPs are are programs that interpret an incomming data structure and produce HTML. To manually ensure that the mapping from that data structure to HTML is correct may require many different page views. For example, one might have session information that causes certain frames to be turned on or off, or there may be incoming flags that cause certain buttons to be greyed or activated.
    - It seems prudent to me to test this code, including all it’s corner cases and boundary conditions. It also seems prudent to put these tests into a regression suite so that they don’t have to be re-run manually every time the session information, or the interface between the middleware and JSPs change.
    - While I agree that it is wise to design your system to minimize the impact of such change, I also think it is wise to test that the impact has been minimized. Doing so econimically requires test automation.

- Ravi Venkataraman 4 days later:
  - - Uncle Bob said, “While I agree that it is wise to design your system to minimize the impact of such change, I also think it is wise to test that the impact has been minimized. Doing so econimically requires test automation.”
    - Actually, I did not talk about minimizing the impact, I talked about eliminating the impact. And that can be designed.
    - The example of frames, too, can be tested without using the UI. I will explain in another post.
    - On another note, I see to it that my business classes never depend on the session parameter; they may have as parameters some bits of information available only through session object, but they never explicitly depend on it. Thus, you can still simulate the functionality without using the container.

- Uncle Bob 4 days later:
  - - I agree that the session object should not be used directly by the business objects. The session is the purview of the presentation portion of the controller code. The same is true of the view attributes. They should be set by the presentation side of the controller in order to direct the detailed behavior of the view.
    - The prolem is that the view still has testable behavior that depends on those view attributes. So long as there is a single if statement or while loop in the JSP that is driven by those view attributes, automated testing of the view behavior is appropriate.

- Ravi Venkataraman 4 days later:
  - - Uncle Bob said, “So long as there is a single if statement or while loop in the JSP that is driven by those view attributes, automated testing of the view behavior is appropriate.”
    - Agreed. But my argument is that even those attributes are data and should not have to be resolved by the JSP. In short, there should be no while blocks in the JSP. The only if blocks should be those that possibly determine what frame to display, what tab to switch to, etc. But even these can be accomplished without if blocks using JavaScript.
    - As I said in my first post, I think of the JSP merely as a means to convey the web page data back to the application server. The application server then uses the data to generate some data and some sort of status. Using this information and a state machine concept, one determines the next action: what JSP page to display, what data and options to display, etc.
    - In such scenarios, there is no need to test the JSP pages independent of the servlet container. There is a clean separation between the View and Model.
    - In my design, there is a JspPage abstract class that serves as a controller. Each JSP page actually maps to one sub-class of this class. When some action is required, the particular sub-class acts as the controller, knows what Java Beans and classes have to be called, and what to do with the response from the JavaBeans class. Basically like what Struts and other tools do, but much simpler and more versatile.
    - In these designs, we do not have to test the JSP page outside the servlet container.

- Uncle Bob 4 days later:
  - - How do you deal with variable length tables and lists if you don’t have loops in your JSPs?
    - How do you deal with conditional messages (like errors) if you don’t have ifstatements in your JSPs?

- Ravi Venkataraman 4 days later:
  - - There are two substitutes for while loops:
    - 1. Have the Jsp ccontroller class spit out a string of HTML and have the JSP say something like: out.println(outputData);
    - 2. Have the data come in as XML and use XSLT to format the data, generating the desired JSP page. This XML/XSLT transformation can be performed programmatically at the back end if desired. Then the JSP reduces to the simple form as shown in #1 above. Of course, other approaches may involve using classes that can convert any Java Object directly to html format, using any desired display format (table, list. etc.) I like XSLT because, although verbose, it gives me some power in conditionally formatting the display. Also, the same data, or subsets thereof, can be displayed in completely different formats.
    - As I’d mentioned earlier, in some cases “if” cases are unavoidable, such as tabs, choice of frames to display, message pop-ups, etc.

- Ravi Venkataraman 4 days later:
  - - I have often thought that Tomcat and other such tools are very heavyweight. All I want is for some mechanism to capture the data present in the HTML form on the page, and the required action, and pass it back to the server.
    - Then, at the server end, we could do whatever is needed and send it back to the web layer. To do this, why do we need many of the features of Tomcat? If provided by the container, Database Connection pooling is useful. Session management through timeout, etc., is useful. But not much else.
    - On a related note, why are the regular web browsers so poor at doing things such as (1) clicking on an html table column to sort by that column; (2) expanding text fields using drag and drop when the area is not sufficient; (3) easy implementation of simple contraints such as numeric only fields, date formats,, and so on. Such features would make it easy to develop web applications.

- Uncle Bob 5 days later:
  - - I’d rather not have my controller class know about HTML syntax. I think that belongs in something like a JSP. (Though Velocity would probably be a better choice.)
    - I can’t say I’m a big fan of XSLT. That code can get pretty twisted, and it really needs to be tested.

- Ravi Venkataraman 5 days later:
  - - OK. You do what works for you.
    - I’ll just keep it simple and cleanly separate my presentation layer from the business layer. I’ll also avoid the ugly and bloated code that results from mixing Java and HTML in a JSP. As a result of the clean separation of layers, I’ll end up writing less code and fewer tests but get a very robust and easily maintainable and extensible architecture.

- Ravi Venkataraman 5 days later:
  - - And by the way, in my architecture it is not the controller class that knows about the HTML syntax. In fact, none of my classes knows about it. I use some of the libraries provided by Java to transform XML to HTML. One could also write custom classes to convert objects to HTML, if one desired. And none of these would be in the controller class!
    - Once the controller class gets the data (XML or Objects) from the server, it invokes the transformation classes and goes ahead presenting the result of the transormation to the view. The controller classes do not do the transform themselves.

- Ilja Preuss 9 days later:
  - - I only skimmed over the article, but you don’t need to manually precompile your JSPs to test them – ServletUnit can do that for you! http://httpunit.sourceforge.net/doc/faq.html#JSP
    - It’s a well hidden feature, but it works great, in my limited experience…
- Criminal Check over 4 years later:
  - - The JspC ant task wants to compile the WHOLE web app for me. And this is such a good information. Keep posting useful and informative articles.

- Criminal Records over 4 years later:
  - - All I want is for some mechanism to capture the data present in the HTML form on the page, and the required action, and pass it back to the server.

- Tenant Screening over 4 years later:
  - - One could also write custom classes to convert objects to HTML, if one desired. And none of these would be in the controller class!