App Engine‎ > ‎

Java Codelab

Maintained by Google Developer Relations
Fall 2011

Objective/Description:
This codelab will demonstrate how to build applications using the Java version of Google App Engine. It is broken down into three sections. 

Part 1 Focuses on the basics which is to build a simple guestbook application. Incidentally, this application is based on the existing "Getting Started" App Engine tutorial for Java, but it also extends upon it a bit as well as offers more detailed explanations. 

Part 2 Focuses on providing a set of examples of how to use the various App Engine services such as email, XMPP.

Part 3 Shows how to use URLFetch, Cron, Task Queues and the Channel API to push browser updates from the server. This is a standalone lab and can be run independently from the first two labs. 


Requirements:
In order to do this lab, you will need:

Part 1: Getting Started

In this first step we'll be creating a new Web project using the Eclipse plugin and building a very simple Java App Engine app that prints out a "Hello World". After creating the app, we'll examine the generated code so we fully understand the different parts of a Java App Engine application. This first step is also a good test to make sure your Eclipse environment is set up correctly for App Engine for Java.

To build a simple "HelloWorld" app (which we'll later convert into a guestbook application), perform the following steps in Eclipse :
  1. Create a new Web Application Project by issuing the following command in Eclipse:
    • File -> New -> Web Application Project 
  2. As the new Web application dialog appears, set the "Project name" to Guestbook and set the "Package" to guestbook
  3. Uncheck "Use Google Web Toolkit," and ensure "Use Google App Engine" is checked.
  4. Click Finish to create the project.
Congratulations! You have just built your first, runnable Google App Engine application!

Before examining the contents of the application project, let's go ahead and run the project. This is easily done by doing the following:
  1. Locate your new project "Guestbook" in the Eclipse navigator and right click:
    • Run As -> Web Application
  2. Alternatively, you can run the application simply by clicking on the project in the navigator (to select it), then clicking on the Eclipse "Run" icon: 

    This will launch your application locally using the App Engine's development server. As the application starts up, you will see the following in the console:
The server is running at http://localhost:8888/
  1. Place this url in your browser and you should see the following:

  2. This is actually a static "index.html" file that is autogenerated. Click on the "Guestbook" link which directs to a generated Java servlet with the path '/guestbook'.
  3. As the servlet is invoked, you will see the simple hello message in the browser:
    Hello, world

Examining the generated Java Web App
Now that you've created and run your first Java App Engine app, let's examine the generated content in more detail.
If you are familiar with standard Java EE Web application development, you will no doubt recognize the similarity of the App Engine App to a Java EE WAR (Web Archive) application structure.

A simple glance at the files from the Eclipse navigator yields:

Some key items to notice:
  • The file and directory structure match a typical Java EE Web app with the exception of an additional file, 'appengine-web.xml'. This file allows you to configure App Engine specific parameters.
  • Also notice that log4j logging files are included in both the src and WEB-INF directory. This is helpful since including helpful logging information can ease debugging once the app is deployed on the server.

Examining the source code of the Java servlet, GuestbookServlet.java, yields:

package guestbook;

import java.io.IOException;
import javax.servlet.http.*;

@SuppressWarnings("serial")
public class GuestbookServlet extends HttpServlet {
  public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
    resp.setContentType("text/plain");
    resp.getWriter().println("Hello, world");
  }
}

As you can see in this simple servlet example, the output ContentType is set to 'text/plain', and a simple text message, 'Hello, World', is sent in the servlet response.

Examining the WEB-INF/web.xml file
In order for this servlet to run, the servlet requires a servlet mapping as defined in the 'WEB-INF/web.xml' file.
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE web-app PUBLIC
 "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
 "http://java.sun.com/dtd/web-app_2_3.dtd">


<web-app xmlns="http://java.sun.com/xml/ns/javaee" version="2.5">
   
<servlet>
       
<servlet-name>guestbook</servlet-name>
       
<servlet-class>guestbook.GuestbookServlet</servlet-class>
   
</servlet>
   
<servlet-mapping>
       
<servlet-name>guestbook</servlet-name>
       
<url-pattern>/guestbook</url-pattern>
   
</servlet-mapping>
   
<welcome-file-list>
       
<welcome-file>/index.html</welcome-file>
   
</welcome-file-list>
</web-app>

As you can see the path, '/guestbook', is associated with the servlet named 'guestbook' which is mapped to the servlet-class guestbook.GuestbookServlet.

Notice also that a welcome-file-list defines the file static file, /index.html, which is what appears when you first visit http://localhost:8888 in a browser.

Examining the WEB-INF/appengine-web.xml
The appengine-web.xml file allows for the setting of App Engine specific configurations.
<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
   
<application></application>
   
<version>1</version> <!-- Configure java.util.logging --> <system-properties> <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/> </system-properties> </appengine-web-app>

In addition to setting the default logging file location, this file contains two important properties:
  • application
    • This property allows you to define the unique name of your application, known as the application-id. Later when you deploy this app to the cloud, your application-id will be the subdomain of the deployed app on appspot.com. i.e. your-application-id.appspot.com.
  • version
    • App Engine supports versioning of your app, so you can use this field to define which version of your app this is.
For more info on the application configuration files for a Java App Engine app, please see: http://code.google.com/appengine/docs/java/config/appconfig.html

This concludes the getting started portion of this codelab. The next steps will demonstrate how to use the various services provided by App Engine

Using App Engine Services - Using the Users Service
App Engine provides an easy to use service that allows you to link authentication to a Google account. As with other App Engine services, you simply need to add the necessary Java packages and use the respective service API code.

To modify the 'Hello World' servlet to first check who the user is logged in, and then supply a login link, and then return back in  an authenticated mode, add the following code changes to the guestbook servlet, GuestbookServlet.java.
  1. Add the following package imports:
    import com.google.appengine.api.users.User;
    import com.google.appengine.api.users.UserService;
    import com.google.appengine.api.users.UserServiceFactory;



  2. Then in the doGet() method of the servlet, replace content with the following code:
    public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
    UserService userService = UserServiceFactory.getUserService();
    User user = userService.getCurrentUser();
        if (user != null) {
            resp.setContentType("text/plain");
            resp.getWriter().println("Hello, " + user.getNickname());
        } else {
            resp.sendRedirect(userService.createLoginURL(req.getRequestURI()));
        }  
    }


The final code  the servlet should now look like:

package guestbook;

import java.io.IOException;
import javax.servlet.http.*;
import com.google.appengine.api.users.User;
import com.google.appengine.api.users.UserService;
import com.google.appengine.api.users.UserServiceFactory;

@SuppressWarnings("serial")
public class GuestbookServlet extends HttpServlet {
  public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
    UserService userService = UserServiceFactory.getUserService();
    User user = userService.getCurrentUser();

    if (user != null) {
        resp.setContentType("text/plain");
        resp.getWriter().println("Hello, " + user.getNickname());
    } else {
        resp.sendRedirect(userService.createLoginURL(req.getRequestURI()));
    }  }
}

Now we'll re-run the app to see the changes. First save the modified file in Eclipse (Ctrl/Command-S). Since we modified the source code of the servlet, we'll have to recycle the local development server. 

  1. To shutdown the server from Eclipse, click on the red button just above the console.
  2. After the server is shutdown, restart it in the same manner as before by clicking on the Run icon, or right-click, Run As-> Web Application.
  3. Return to the browser at location: http://localhost:8888 and access the guestbook servlet again.
  4. You should see the following login screen rendered by the local development server:
  5. As you click on the Log In button, you will be redirected back to the app, but now that you have authenticated, it will respond and display your user id in the output.

    Hello, test@example.com
  6. Obviously this is not a real id, but when this app is deployed, it will attach to Google's actual authentication service which means that when a user logs in using their real Google/Gmail id, they will see their ID in the response from the servlet.
Next, we'll move on to creating forms in a Web page and processing the data submitted from a form. 


Using JSPs in App Engine
Similar to other Java EE web apps, App Engine supports JavaServer Pages (JSP). Like other Web templating technologies, JSP allow you to use HTML and insert only portions of Java code into the page for dynamically rendered content.

As a first example of a JSP, we'll create a comparable JSP page that will do the same thing as the Guestbook servlet.

To create a JSP that displays the current user, do the following:
  1. In the Eclipse navigator, locate the war folder and then create a JSP file: right-click the war folder in the navigator and then select New -> File.
  2. Name the file guestbook.jsp and click Finish.
  3. After the file is created, open the file in Eclipse's Text Editor (right-click Open with -> Text Editor)
    • Note: The new file may be opened in another editor on your system outside of Eclipse upon creation. If this happens, simply close the other editor and re-open the file in Eclipse using the above step.
  4. Then add the following content to the file.
    <%@ page contentType="text/html;charset=UTF-8" language="java" %>
    <%@ page import="com.google.appengine.api.users.User" %>
    <%@ page import="com.google.appengine.api.users.UserService" %>
    <%@ page import="com.google.appengine.api.users.UserServiceFactory" %>
    
    <html>
      <body>
    
    <%
        UserService userService = UserServiceFactory.getUserService();
        User user = userService.getCurrentUser();
        if (user != null) {
    %>
    <p>Hello, <%= user.getNickname() %>! (You can
    <a href="<%= userService.createLogoutURL(request.getRequestURI()) %>">sign out</a>.)</p>
    <%
        } else {
    %>
    <p>Hello!
    <a href="<%= userService.createLoginURL(request.getRequestURI()) %>">Sign in</a>
    to include your name with greetings you post.</p>
    <%
        }
    %>
    
      </body>
    </html>

  5. Save the guestbook.jsp file.
  6. Now run the page by browsing to http://localhost:8888/guestbook.jsp (make sure the local server is still running)
  7. You'll see that the following appears now in the browser:

    Hello, test@example.com! (You can sign out.)

  8. Click on the sign out link if you want to sign out and re-sign in again.

Working with Forms
Now that we've created both servlets and JSPs in App Engine, it's now time to finish this guestbook example and create a Web form along with server-side code to process form parameters.
  1. In the JSP page, add the following code to just above the closing </body> tag:
     <form action="/sign" method="post">
        <div><textarea name="content" rows="3" cols="60"></textarea></div>
        <div><input type="submit" value="Post Greeting" /></div>
     </form>

Your JSP now contains a Web form, but in order for it to truly work, a server-side handler must be created in order to handle the incoming http POST coming from the JSP page. This can be done by adding a doPost() method to our existing servlet, but instead we'll create a new servlet and have it correspond to the "/sign" URL referred to by form's action property.

Creating a server-side form handler as a Servlet
To create our '/sign' handler, we'll create a new Java Servlet by doing the following:
  1. Create a new Java class, using File -> New -> Class.
  2. In the Java class creation dialog, specify the Name: SignGuestbookServlet with the same package, guestbook.
  3. Optional:
    • Before clicking Finish, you can also specify the Superclass: as javax.servlet.http.HttpServlet. Note: you can click on the Browse button on the right to select the superclass by typing in "httpservlet" as a search criteria. This will locate the correct superclass. (This is an optional step, as we'll be replacing this code by copying and pasting a completed example below - but is helpful in order to understand how to create a subclass - such as a Java Servlet)
    • Upon clicking Finish, this will generate a Java class that "extends HttpServlet". 
  4. After creating your new Java class you'll notice that it does not have any methods. 
    • If you're an experienced Eclipse user, you can select Source -> Override/Implement Methods... and then select the doPost(...) option to generate an empty doPost() method. This step is mainly meant to explain how to override or implement methods in Eclipse. For this example though, we'll just use the example code below.
  5. In your new Java class, copy and replace the entire content with this code:
package guestbook;

import java.io.IOException;
import java.util.logging.Logger;
import javax.servlet.http.*;
import com.google.appengine.api.users.User;
import com.google.appengine.api.users.UserService;
import com.google.appengine.api.users.UserServiceFactory;

public class SignGuestbookServlet extends HttpServlet {
   
private static final Logger log = Logger.getLogger(SignGuestbookServlet.class.getName());

   
public void doPost(HttpServletRequest req, HttpServletResponse resp)
               
throws IOException {
       
UserService userService = UserServiceFactory.getUserService();
       
User user = userService.getCurrentUser();
       
String content = req.getParameter("content");
       
if (content == null) {
            content
= "(No greeting)";
       
}
       
if (user != null) {
            resp.getWriter().println
("Greeting posted by user: " + user.getNickname() + ": " + content);
       
} else {
            resp.getWriter().println
("Greeting posted anonymously: " + content);
       
}
    }
}

Although we've created a Java class extends HttpServlet, in order for this code to be invoked via a specific URL, we need to edit the WEB-INF/web.xml and add a servlet entry that defines both the name of the servlet as well as the servlet mapping which provides a URL pattern to execute.

  1. Open the WEB-INF/web.xml and add the following above the closing </webapp> tag:
    <servlet> <servlet-name>sign</servlet-name> <servlet-class>guestbook.SignGuestbookServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>sign</servlet-name>   <url-pattern>/sign</url-pattern> </servlet-mapping>

You should now have a functioning server-side form handler in this new servlet, which responds to an HTTP Post to the url: '/sign'.

To test the app:
  1. Save all, and restart the application again. Once the server is up, point your browser again to http://localhost:8888/guestbook.jsp
  2. Notice the difference when you authenticate, and when you don't authenticate.
At this point you have done the typical "Hello World" for all Web application development which is to write both the front end code that renders a form, as well as write the server-side code that responds to a form POST. In the previous example, the servlet responding to an HTTP Post to the URL, "/sign", simply extracts the value passed in the form parameter, "content", and simply prints it back out to the browser.

Our next step is to persist this guestbook data into App Engine's Datastore.

Persisting data in the Datastore
The App Engine datastore is one of several services provided by App Engine with two APIs: a standard API, and a low-level API. App Engine's standard data API includes support for both: Java Data Objects (JDO) and Java Persistence API (JPA). These interfaces are provided by DataNucleus Access Platform, an open source implementation of several Java persistence standards, with an adapter for the App Engine datastore.
Using the standard APIs (JDO/JPA) makes it easier to port your applications to other Java platforms, however for simplicity, we'll assume that we would not need to port this application to another Java server environment, so we will use the low-level API for data storage in the next steps.

In the Java servlet, SignGuestbookServlet.java, we'll replace the doPost() method with the following code that stores greetings in the datastore. 

public void doPost(HttpServletRequest req, HttpServletResponse resp)
        throws IOException {
    UserService userService = UserServiceFactory.getUserService();
    User user = userService.getCurrentUser();

    String guestbookName = req.getParameter("guestbookName");
    Key guestbookKey = KeyFactory.createKey("Guestbook", guestbookName);
    String content = req.getParameter("content");
    Date date = new Date();
    Entity greeting = new Entity("Greeting", guestbookKey);
    greeting.setProperty("user", user);
    greeting.setProperty("date", date);
    greeting.setProperty("content", content);

    DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
    datastore.put(greeting);
    
    resp.sendRedirect("/guestbook.jsp?guestbookName=" + guestbookName);
}

In order for this to compile cleanly, we'll have the following import statements
import java.io.IOException;
import java.util.Date;
import java.util.logging.Logger;
import javax.servlet.http.*;

import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.Key;
import com.google.appengine.api.datastore.KeyFactory;
import com.google.appengine.api.users.User;
import com.google.appengine.api.users.UserService;
import com.google.appengine.api.users.UserServiceFactory;

The complete source code for the servlet will now be. (You may copy this code directly into your SignGuestbookServlet.java class.)
package guestbook;

import java.io.IOException;
import java.util.Date;
import java.util.logging.Logger;
import javax.servlet.http.*;

import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.Key;
import com.google.appengine.api.datastore.KeyFactory;
import com.google.appengine.api.users.User;
import com.google.appengine.api.users.UserService;
import com.google.appengine.api.users.UserServiceFactory;

public class SignGuestbookServlet extends HttpServlet {
    private static final Logger log = Logger.getLogger(SignGuestbookServlet.class.getName());

    public void doPost(HttpServletRequest req, HttpServletResponse resp)
            throws IOException {
        UserService userService = UserServiceFactory.getUserService();
        User user = userService.getCurrentUser();

        String guestbookName = req.getParameter("guestbookName");
        Key guestbookKey = KeyFactory.createKey("Guestbook", guestbookName);
        String content = req.getParameter("content");
        Date date = new Date();
        Entity greeting = new Entity("Greeting", guestbookKey);
        greeting.setProperty("user", user);
        greeting.setProperty("date", date);
        greeting.setProperty("content", content);

        DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
        datastore.put(greeting);
        
        resp.sendRedirect("/guestbook.jsp?guestbookName=" + guestbookName);
    }
}

Storing the submitted greetings
The low-level Datastore API for Java provides a schemaless interface for creating and storing entities. The low-level API does not require entities of the same kind to have the same properties, nor for a given property to have the same type for different entities. The following code snippet constructs the Greeting entity in the same entity group as the guestbook to which it belongs:
        Entity greeting = new Entity("Greeting", guestbookKey);
        greeting
.setProperty("user", user);
        greeting
.setProperty("date", date);
        greeting
.setProperty("content", content);

After we construct the entity, we instantiate the datastore service, and put the entity in the datastore:

        DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
        datastore
.put(greeting);


Updating the JSP

We'll now update the JSP page to both query the contents of the datastore as well as retain the form to add new guestbook entries.
Edit the war/guestbook.jsp and completely replace the entire contents of your JSP with the content below.
<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<%@ page import="java.util.List" %>
<%@ page import="com.google.appengine.api.users.User" %>
<%@ page import="com.google.appengine.api.users.UserService" %>
<%@ page import="com.google.appengine.api.users.UserServiceFactory" %>
<%@ page import="com.google.appengine.api.datastore.DatastoreServiceFactory" %>
<%@ page import="com.google.appengine.api.datastore.DatastoreService" %>
<%@ page import="com.google.appengine.api.datastore.Query" %>
<%@ page import="com.google.appengine.api.datastore.Entity" %>
<%@ page import="com.google.appengine.api.datastore.FetchOptions" %>
<%@ page import="com.google.appengine.api.datastore.Key" %>
<%@ page import="com.google.appengine.api.datastore.KeyFactory" %>

<html>
  <head>
    <link type="text/css" rel="stylesheet" href="/stylesheets/main.css" />
  </head>

  <body>

<%
    String guestbookName = request.getParameter("guestbookName");
    if (guestbookName == null) {
        guestbookName = "default";
    }
    UserService userService = UserServiceFactory.getUserService();
    User user = userService.getCurrentUser();
    if (user != null) {
%>
<p>Hello, <%= user.getNickname() %>! (You can
<a href="<%= userService.createLogoutURL(request.getRequestURI()) %>">sign out</a>.)</p>
<%
    } else {
%>
<p>Hello!
<a href="<%= userService.createLoginURL(request.getRequestURI()) %>">Sign in</a>
to include your name with greetings you post.</p>
<%
    }
%>

<%
    DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
    Key guestbookKey = KeyFactory.createKey("Guestbook", guestbookName);
    // Run an ancestor query to ensure we see the most up-to-date
    // view of the Greetings belonging to the selected Guestbook.
    Query query = new Query("Greeting", guestbookKey).addSort("date", Query.SortDirection.DESCENDING);
    List<Entity> greetings = datastore.prepare(query).asList(FetchOptions.Builder.withLimit(5));
    if (greetings.isEmpty()) {
        %>
        <p>Guestbook '<%= guestbookName %>' has no messages.</p>
        <%
    } else {
        %>
        <p>Messages in Guestbook '<%= guestbookName %>'.</p>
        <%
        for (Entity greeting : greetings) {
            if (greeting.getProperty("user") == null) {
                %>
                <p>An anonymous person wrote:</p>
                <%
            } else {
                %>
                <p><b><%= ((User) greeting.getProperty("user")).getNickname() %></b> wrote:</p>
                <%
            }
            %>
            <blockquote><%= greeting.getProperty("content") %></blockquote>
            <%
        }
    }
%>

    <form action="/sign" method="post">
      <div><textarea name="content" rows="3" cols="60"></textarea></div>
      <div><input type="submit" value="Post Greeting" /></div>
      <input type="hidden" name="guestbookName" value="<%= guestbookName %>"/>
    </form>

  </body>
</html>

Now that we have complete both the server-side code to insert a "greeting" into the datastore, as well as front end code in a JSP to query the datastore for any existing greetings, we can now run the application!

  1. As before, to make sure we load the latest compiled Java classes, you'll need to make sure to Save all and then recycle the server by clicking on the stop button above the console and then restarting the guestbook app as before.
  2. As the app starts up again, you can re-visit http://localhost:8888/guestbook.jsp
  3. Enter a comment and submit the form. You should then see the comment displayed above the form.
  4. Feel free to enter multiple comments while both signed in and not.
Congratulations! You have just built your first Google App Engine application!


Editing the local datastore using the SDK Console (optional)

As a quick additional step in this codelab, it is very easy to view and edit the data in the local datastore. This is done with the SDK Console. We'll now delete the comments in our local datastore using the SDK console.

  1. While the local server is still running, go to the location: http://localhost:8888/_ah/admin/datastore
    You should see the following:

  2. Click on the button "List Entities". This will list what you currently have stored for the Greeting entity in the datastore.
  3. You should then see the current datastore greeting entities. You can now delete them by checking the checkbox on each line and clicking the "Delete" button at the bottom.
  4. Now if you return to the guestbook page: http://localhost:8888/guestbook.jsp (and hit refresh) you will see that the deleted comments no longer appear in the page.

Deploying your guestbook application 
Now that you have a complete application that is runnable locally, you can now deploy it to the cloud. This is a relatively simple step, but it does require you to have a Google account. If you have yet to create an App Engine account, please follow the steps in the "Creating your App Engine Account" codelab also contained on this site.

As you follow the steps in this codelab you will also be creating a new, empty placeholder application, so think of a unique name for your guestbook application, such as 'myuniqueguestbookapp' , so that it can be accessed on the Internet at the location: myuniqueguestbookapp.appspot.com.

Once you've successfully created a uniquely named application on the App Engine dashboard, you will then be able to apply that unique application identifier (app id) to your local guestbook app's war/WEB-INF/appengine-web.xml file in the following manner:

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>myuniqueguestbookapp</application>
<version>1</version>
<!-- Configure java.util.logging -->
<system-properties>
<property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
</system-properties>
</appengine-web-app>

At this point you can save all, and deploy the app to the cloud by:

  1. Selecting the guestbook application in the navigator, and then right-clicking Google -> Deploy to App Engine.



  2. After selecting this, a dialog will appear where you will have to enter your App Engine (Google) account credentials in order to authorize the deployment. Once entered, your application will be deployed to the cloud for all to see at: http://myuniqueguestbookapp.appspot.com

This completes part 1 of the Java App Engine codelab. The second part will provide further examples of how to work with the various services of App Engine.

Part 2: Using App Engine Services Email & XMPP

This second part of the Code Lab builds on the first part but each segment is independent and does not require each other to successfully complete. In this part, a series of independent exercises are provided which show how to write code using the various services of App Engine, such as email, XMPP.


Using Email in App Engine

Sending an email is one of the most common things a Web application does such as when somebody new registers for the app, etc. This next exercise shows how to create and send an email using App Engine's email service.

We'll start by providing a simple method that can be used to send an email from any Java class. This method is called sendEmail() and accepts three String arguments, recipient, subject, and text.

  
  public void sendEmail(String recipient, String subject, String text) {
    Properties props = new Properties();
    Session session = Session.getDefaultInstance(props, null);
    try {
      Message msg = new MimeMessage(session);
      msg.setFrom(new InternetAddress("YOUREMAILGOESHERE"));
      msg.addRecipient(Message.RecipientType.TO,
                             new InternetAddress(recipient));
      msg.setSubject(subject);
      msg.setText(text);
      Transport.send(msg);
    } catch (AddressException e) {
         // ...
    } catch (MessagingException e) {
         // ...
    }
  }  
  
Notice that this method has "YOUREMAILGOESHERE" in it? This is where you would supply your email, or at least the email that is associated with the App Engine account that contains this application. If you put a random email in this location, the email will not be sent.  

In order for this method to compile properly, you must add the following imports to its container class:

import java.util.Properties;
import javax.mail.MessagingException;
import javax.mail.Message;
import javax.mail.Session;
import javax.mail.Transport;
import javax.mail.internet.AddressException; 
import javax.mail.internet.InternetAddress; 
import javax.mail.internet.MimeMessage;   

One possible usage that you could try would be to add this method to the servlet which processes the posted comment, so when the comment is posted and inserted into the datastore,  it could also send an email with the commented text.

So returning back to the SignGuestbookServlet class, just before you post a new comment to the datastore, you could send it out as an email using this code:

public void doPost(HttpServletRequest req, HttpServletResponse resp)
                throws IOException {
        UserService userService = UserServiceFactory.getUserService();
        User user = userService.getCurrentUser();

        String content = req.getParameter("content");
        Date date = new Date();
        Greeting greeting = new Greeting(user, content, date);
        
        sendEmail("RECIPIENTEMAIL", "Someone posted a comment on your guestbook.", greeting.getContent());

        PersistenceManager pm = PMF.get().getPersistenceManager();
        try {
            pm.makePersistent(greeting);
        } finally {
            pm.close();
        }

        resp.sendRedirect("/guestbook.jsp");
 }


    Notice that you'll need to change RECIPIENTEMAIL with an actual email address that you'd wish to send you msg to.

To test this new code you'll have to save, compile, and redeploy to the cloud as it is not possible to send email from a local SDK instance.

Once deployed, when anyone posts a comment on your guestbook app, it will send you a message containing the posted comment.


Using XMPP in App Engine - Building your own Chatbot

With App Engine's XMPP support, you can build a chat enabled service on your app. Here's some example code that you can add to your application. It is a servlet that serves as an XMPP chatbot to the guestbook application. It will respond to a Jabber client and list all of the comments enter so far by people.

package guestbook;

import java.io.IOException;

import java.util.List;
import javax.jdo.PersistenceManager;
import javax.servlet.ServletException;
import javax.servlet.http.*;

import guestbook.PMF;
import guestbook.Greeting;

import com.google.appengine.api.xmpp.JID;
import com.google.appengine.api.xmpp.Message;
import com.google.appengine.api.xmpp.MessageBuilder;
import com.google.appengine.api.xmpp.MessageType;

import com.google.appengine.api.xmpp.SendResponse;

import com.google.appengine.api.xmpp.XMPPService;
import com.google.appengine.api.xmpp.XMPPServiceFactory;

@SuppressWarnings("serial")
public class GreetingChatbot extends HttpServlet {

  @Override
  protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException,
      IOException {

    XMPPService xmpp = XMPPServiceFactory.getXMPPService();
   
    Message message = xmpp.parseMessage(req);
    //
    JID fromJid = message.getFromJid();
    String body = message.getBody();

    String respMsg = null;

    if (body.equals("/list")) {
      respMsg = getGreetingList();
    } else if (body.equals("/help")) {
      respMsg = "Welcome to the Guestbook Chatbot!\nThe following commands are supported: \n /list \n /help";
    } else {
      respMsg = "Command '" + body + "' not supported! \nEnter '/help' for list of commands.";
    }

    // Send response..
    JID tojid = new JID(fromJid.getId());

    Message msg = new MessageBuilder().withRecipientJids(tojid).withBody(respMsg).build();
    
    boolean messageSent = false;
    xmpp = XMPPServiceFactory.getXMPPService();
    if (xmpp.getPresence(tojid).isAvailable()) {
      SendResponse status = xmpp.sendMessage(msg);
      messageSent = (status.getStatusMap().get(tojid) == SendResponse.Status.SUCCESS);
    }

  }

  public String getGreetingList() {

    PersistenceManager pm = PMF.get().getPersistenceManager();

    String query = "";
    query = "select from " + Greeting.class.getName();
    List<Greeting> greetings = (List<Greeting>) pm.newQuery(query).execute();

    String greetinglist = "Greetings list:";

    for (Greeting g : greetings) {
      greetinglist += "\n" + g.getAuthor() + " posted:\n";
      greetinglist += g.getContent()
          + "\n";
    }
    pm.close();
    return greetinglist;
  }
}  

To explain the code a bit, this is a chatbot that can respond to a Jabber client's text input. In this example it takes a simple set of commands, /help and /list. The /help command simply cause the bot to list both of these commands. :-)
The /list command causes the bot to repond back with all of the comments(greetings) from the Guestbook in the datastore.
That's it!

To add this XMPP functionality to your guestbook app, you'll need to do the following:

  1. Create a new Java class GreetingChatbot in the same guestbook package.
  2. Entirely replace the generated code with the code above.
  3. Next, you'll need to set up a Java servlet mapping so that a special URL path is mapped to this servlet. Edit you web.xml and add the mapping as:
    <servlet>
        <servlet-name>xmppreceiver</servlet-name>
        <servlet-class>guestbook.GreetingChatbot</servlet-class>
    </servlet>
    <servlet-mapping>
        <servlet-name>xmppreceiver</servlet-name>
        <url-pattern>/_ah/xmpp/message/chat/</url-pattern>
    </servlet-mapping>
  4. Notice the url-pattern is "/_ah/xmpp/message/chat/". This must be exactly in this format.
  5. Save and close the web.xml
  6. Now edit the appengine-web.xml file and add the following:
    <inbound-services>
        <service>xmpp_message</service>
    </inbound-services>
  7. Save, close and deploy your app to the cloud. Note: You must deploy it, as you can't test XMPP locally!

To test this app, you must: 
  1. Once deployed, you you can use a GMail Gtalk window. To open up a connection to your new guestbook bot, do the following:
  2. Click on Add Contact in Gtalk.
  3. Then add the address of your guestbook application as: myuniqueguestbookapp@appspot.com
    Important: notice that the '@' symbol separates the app name from 'appspot.com'.
  4. That's it! You should be able to initiate a connection and begin making commands.


Congratulations! You have completed parts 1 and 2 of the Java App Engine Codelab!





(New: Created for Google IO 2011)

Part 3: Using App Engine Services UrlFetch, Cron, Channel API, and Task Queues


This third part is a stand alone module which will show you how to use the services:
  • UrlFetch - to fetch data from another Internet source
  • Cron - to run recurring offline jobs
  • Channel API - to push updates to browser windows without needing to refresh (aka Comet)
  • Task Queues - To run a long offline batch process
The complete application these exercise build to is an application that fetches a news feed and then pushes the titles, headlines and descriptions to browsers via the Channel API.

Note: the complete zipped version of this application is available here.
(instructions on how to run the complete application are located at the end of this section)

Using URLFetch to retrieve an XML feed


Creating a new project and building a servlet to fetch feeds

For this portion of the codelab, we will start with a fresh new Eclipse project.
  1. (as before) In Eclipse, select File->New->Project...Google->Web Application Project and click Next.
  2. Name the project 'NewsClientChannelAPI' and specify 'com' as the default package.
    Important: Make sure to deselect the "Use Google Web Toolkit" option. 
  3. Then click Finish to generate the project.
  4. You will now update the auto-generated servlet, by copying and pasting it into a new servlet. 
    1. In the Project Explorer window locate the generated servlet source file: src-> com. NewsClientChannel..Servlet.java.
    2. Copy the file (Right- Click on the file in the explorer, and then select Copy) (Command/Control-C also works)
    3. Paste the contents into the same location. (Command/Control-V)
    4. In the dialog that appears: "Enter a new name for..", enter the text "FetchNewsServlet" and click "OK".
    5. You should now have a servlet name "FetchNewsServlet.java" in the project navigator.
We now have a new Java class, but in order to make it accessible via a URL, we must edit the web.xml and add a mapping for this new servlet class. 
  1. In the project navigator, locate the web.xml file. war->WEB-INF->web.xml
  2. Open the file in the editor. You'll notice the servlet entry for the auto-generated servlet.
  3. Copy and past this entire entry, into a new servlet entry and mapping. Then edit the contents with new information of your new servlet with the following:

    <servlet>
       <servlet-name>FetchNews</servlet-name>
       <servlet-class>com.FetchNewsServlet</servlet-class>
    </servlet>
    <servlet-mapping>
         <servlet-name>FetchNews</servlet-name>
         <url-pattern>/cron/fetchNews</url-pattern>
    </servlet-mapping>


  4. You'll notice the url-pattern includes /cron. This is to allow a cron job run this fetch service periodically to grab fresh news entries. (We'll add the actual cron entry later.)
Now we must alter the servlet's code to fetch an XML feed with the URLFetch service.
  1. Open the servlet file com.FetchNewsServlet in the editor.
  2. Using the documentation example of 'URLFetch' - implemented with java.net.URL, You can copy the code from the example shown here: http://code.google.com/appengine/docs/java/urlfetch/usingjavanet.html#Simple_Requests_With_URLs
    into your servlet's
    doGet() method.
  3. You'll then need to alter it so that it fetches a news feed. You can use the following code:

      public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {

        resp.setContentType("text/html");
        resp.getWriter().println("Getting feed...<br/>");

        String googleFeed = "http://news.google.com/news?ned=us&topic=h&output=rss";
        String totalFeed = "";

        try {
          URL url = new URL(googleFeed);
          BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()));
          String line = null;
         while ((line = reader.readLine()) != null) {
            totalFeed += line;
            System.out.println(totalFeed );
          }

          reader.close();
          //parseFeedandPersist(totalFeed, resp);

        } catch (MalformedURLException e) {
          // ...
        } catch (IOException e) {
          // ...
        }
    }



  4. In order to compile this code you'll need to import the following packages:

    import java.io.BufferedReader;
    import java.io.IOException;
    import java.io.InputStreamReader;
    import java.net.MalformedURLException;
    import java.net.URL;

    import javax.servlet.http.*;


  5. You'll notice that there's a method parseFeedandPersist() that is commented out. We'll add that method shortly, but for now, you can try running this servlet. As before, to run an App Engine project locally, first click on the project in the navigator, then select Run->Run as Web Application.
  6. Alternatively, you can just click the project and click on the run icon at the top of the Eclipse. 
  7. Once the app is running in the console, you can point your browser to:
    http://localhost:8888/cron/fetchNews

  8. Upon accessing this URL, you will not see the feed data in the browser yet, but will see XML feed data in the console. We will now add code to parse the XML data, display in the browser as well as persist it for later use in the application.
Parsing the XML and persisting in the datastore

At this point our code simply fetches raw XML data, but in order for it to be useful, we need to parse the data. For this, we will add two new methods to the servlet: parseFeedandPersist(), parseXML().

The first method parseFeedandPersist(), will take a String argument containing the fetched feed data, parse it, and for now simply display it in the browser.

  1. You can now uncomment the method call parseFeedandPersist() and add the following code to your servlet.

     

      private void parseFeedandPersist(String feedContent, HttpServletResponse resp) throws IOException {        
        Document doc = parseXml(feedContent);

        NodeList titles = null;
        NodeList links = null;
        NodeList descriptions = null;

        if (doc != null) {
          resp.getWriter().println("node: " + doc.getDocumentElement().getNodeName());
          titles = doc.getDocumentElement().getElementsByTagName("title");
          links = doc.getDocumentElement().getElementsByTagName("link");
          descriptions = doc.getDocumentElement().getElementsByTagName("description");
        } else {
          resp.getWriter().println("no input/bad xml input. please send parameter content=<xml>");
        }

       
    //persistNewsData(titles, links, descriptions);

        resp.getWriter().println(
    "<h3>Successfully fetched the following news from feed.</h3>");
        for (int i = 1; i < titles.getLength(); i++) {
          resp.getWriter().println("<br/>Title: " + titles.item(i).getTextContent());
          resp.getWriter().println(
              "<br/>Link: <a href=\"" + links.item(i).getTextContent() + "\">"
                  + titles.item(i).getTextContent() + "</a>");
          if (descriptions.item(i) != null){
            resp.getWriter().println("<br/>Description: " + descriptions.item(i).getTextContent());        
          }

          resp.getWriter().println("<br/><br/>");
        }
      }


     You'll notice in this code that for now the method call persistNewsData() is commented outBefore adding that method, we'll now stop and rerun the app to see this change.

  2. Now we'll add the parseXML() method.
     

      private static Document parseXml(String strXml) {
        Document doc = null;
        String strError;
        try {
          DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
          DocumentBuilder db = dbf.newDocumentBuilder();

          StringReader reader = new StringReader(strXml);
          InputSource inputSource = new InputSource(reader);

          doc = db.parse(inputSource);

          return doc;
        } catch (IOException ioe) {
          strError = ioe.toString();
        } catch (ParserConfigurationException pce) {
          strError = pce.toString();
        } catch (SAXException se) {
          strError = se.toString();
        } catch (Exception e) {
          strError = e.toString();
        }
        
    return null;
      }






  3. Verify the following imports:

    import java.io.BufferedReader;
    import java.io.IOException;
    import java.io.InputStreamReader;
    import java.io.StringReader;
    import java.net.MalformedURLException;
    import java.net.URL;

    import
    javax.jdo.PersistenceManager;
    import javax.servlet.http.*;

    import
    javax.xml.parsers.DocumentBuilder;
    import javax.xml.parsers.DocumentBuilderFactory;
    import javax.xml.parsers.ParserConfigurationException;

    import
    org.w3c.dom.Document;
    import org.w3c.dom.NodeList;
    import org.xml.sax.InputSource;
    import org.xml.sax.SAXException;

    import
    com.google.appengine.api.datastore.Text;


  4. Make sure to stop the currently running app, save the servlet file, and then restart the application.
  5. Upon accessing the app at http://localhost:8888/cron/fetchNews in the browser you should now see the news feed data.


Persisting Feed Data
We will now add the method PersistNewsData(), but for this method to work properly we will have to add a bit of code that allows us to store feed data, so we will now create a new class NewsItem.java and a factory class for storing data (PMF.java).
  1. Add a new Java class (File->New->Class)
  2. In the new Java Class dialog, specify a package of com.model and name the class, PMF.
  3. Open the new PMF class in the editor and replace with the following code:

    package com.model;

    import javax.jdo.JDOHelper;
    import javax.jdo.PersistenceManagerFactory;

    public final class PMF {
        private static final PersistenceManagerFactory pmfInstance =
            JDOHelper.getPersistenceManagerFactory("transactions-optional");

        private PMF() {}

        public static PersistenceManagerFactory get() {
            return pmfInstance;
        }
    }

     
  4. We will now create the class to contain the feed data (File->New->Class).
  5. In the new Class dialog again specify com.model as the package and name the class NewsItem. (Click Finish)
  6. In the java class, NewsItem, replace the contents with the following code:

    package com.model;
    import javax.jdo.annotations.IdGeneratorStrategy;
    import javax.jdo.annotations.PersistenceCapable;
    import javax.jdo.annotations.Persistent;
    import javax.jdo.annotations.PrimaryKey;

    import com.google.appengine.api.datastore.Key;
    import com.google.appengine.api.datastore.Text;

    @PersistenceCapable
    public class NewsItem {
      @PrimaryKey
      @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
      private Key key;
      private String title;
      private String link;
      private Text description;

      public Key getKey() {
        return key;
      }

      public void setKey(Key key) {
        this.key = key;
      }

      public String getTitle() {
        return title;
      }

      public void setTitle(String title) {
        this.title = title;
      }

      public String getLink() {
        return link;
      }

      public void setLink(String link) {
        this.link = link;
      }

      public Text getDescription() {
        return description;
      }

      public void setDescription(Text description) { 
        this.description = description;
      }

    }


  7. Save all and ensure that this code compiles successfully.
  8. Returning to the FetchNewsServlet class add the new method persistNewsData():

    private void persistNewsData(NodeList titles, NodeList links, NodeList descriptions) {
        // Persists news data feed.
        // Also clears out previously persisted news feed data

        PersistenceManager pm = PMF.get().getPersistenceManager();
        javax.jdo.Query query = pm.newQuery(NewsItem.class);
        Long res = query.deletePersistentAll();

        System.out.println("Datastore deleted  " + res + "records");

        pm = PMF.get().getPersistenceManager();

        try {
          for (int i = 1; i < titles.getLength(); i++) {
            NewsItem ni = new NewsItem();
            ni.setTitle(titles.item(i).getTextContent());
            ni.setLink(links.item(i).getTextContent());
            if (descriptions.item(i) != null) {
              ni.setDescription(new Text(descriptions.item(i).getTextContent()));
            } 
            pm.makePersistent(ni);
          }
        } finally {
          pm.close();
        }
      }


  9. You may notice that this code first cleans the current contents of NewsItem in the datastore using query.deletePersistentAll();
    This ensure that the datastore always has fresh news content. 
  10. At this point you should make sure you have the following imports.

    import java.io.BufferedReader;
    import java.io.IOException;
    import java.io.InputStreamReader;
    import java.io.StringReader;
    import java.net.MalformedURLException;
    import java.net.URL;

    import javax.jdo.PersistenceManager;
    import javax.servlet.http.*;
    import javax.xml.parsers.DocumentBuilder;
    import javax.xml.parsers.DocumentBuilderFactory;
    import javax.xml.parsers.ParserConfigurationException;

    import
    org.w3c.dom.Document;
    import org.w3c.dom.NodeList;
    import org.xml.sax.InputSource;
    import org.xml.sax.SAXException;

    import
    com.google.appengine.api.datastore.Text;
    import com.model.NewsItem;
    import com.model.PMF;


  11. Save all and verify a clean compile.
  12. In the parseFeedandPersist() method, you can now uncomment the line:

      // persistNewsData(titles, links, descriptions);

  13. Save all a verify a clean compile.
  14. You can now stop and restart the app.
  15. Access the app again at: http://localhost:8888/cron/fetchNews
  16. In addition to displaying the feed content in the browser as before, you should also see the new data in the datastore. Access the local SDK console at: http://localhost:8888/_ah/admin

    Click on the DataStore Viewer link on the left to see the new entries of news in the datastore entity NewsItem:


  17. At this point we can turn on the Cron job in order to have this execute every few minutes in order to ensure that fresh news is always stored in the datastore. To do this we simply add a cron.xml file as a peer to the appengine-web.xml file under war/WEB-INF/ . 

  18. Create a new file by locating the parent directory war/WEB-INF in the project navigator and right-click and select New->File. In the New File dialog, name the file "cron.xml" and verify that it will be created in the war/WEB-INF/ location and click FInish.

  19. Upon creation, the file will be opened in the editor and will be empty. Add the following cron entry.

    <?xml version="1.0" encoding="UTF-8"?>
    <cronentries>
      <cron>
        <url>/cron/fetchNews</url>
        <description>Repopulate news headlines cache every 5 minutes</description>
        <schedule>every 5 minutes</schedule>
      </cron>
    </cronentries>



  20. You may also want to secure the URL /cron/fetchNews once the app is deployed. This can be done by adding the following security constraint the main webapp config file: war/WEB-INF/web.xml. At this point it is commented out, but when you plan to deploy the app, you can uncomment.
     
    <!-- This should be uncommented when app is deployed --> 
    <!-- 
      <security-constraint>
            <web-resource-collection>
               <web-resource-name>restricted urls</web-resource-name>
                <url-pattern>/cron/*</url-pattern>
            </web-resource-collection>
            <auth-constraint>
                <role-name>admin</role-name>
            </auth-constraint>
        </security-constraint>
     -->


  21. Save all and verify a clean compile. Optional: At this point you may wish to restart the app and verify that the cron job is registered correctly and even let it run locally. 
  22. Now that we have a functioning Web app that fetches a news feed every five minutes, you can deploy the app to the cloud. As before, you will have to create a new app at http://appengine.google.com. Once you have chosen a new app id, you can then set it in your application by either editing the appengine-web.xml file directly using:

    <
    application>your-app-id</application>
    <version>1</version
    >

    Or by clicking the project in the navigator, right-clicking and selecting Google->App Engine Settings.... From this App Engine settings page, you can specify the Application ID (and click OK).

  23. Once your Application ID is specified, you can deploy your application. (Click on the project, right-click and select Google->Deploy to App Engine).
  24. Once deployed, you can first force the fetch of the news feed by going to http://your-app-id.appspot.com/cron/fetchNews. You should then see new NewsItem entity data in the datastore by visiting your app's dashboard and clicking on Datastore Viewer link on the left. You can also check that your cron job has been properly registered by clicking on the Cron Jobs link on the left.

Using the Channel API - setting up a client and server

This section will involve setting up a communication channel from a JavaScript client and a Java servlet using App Engine's Channel API.  This will allow the server to send data down to the client at any time and the client can update it's UI based on these changes. Later in the exercise, we will push down news feed data to the client, but our first step will simply be to write the code to establish the link between the client and server, as well as persisting the client information so the application can maintain a registry of registered client applications from which to push updates to.

Building a servlet to create a Channel API 'token' to pass back to a JavaScript client.
  1. Create a new servlet called TokenServlet.java. As before, the easiest way to creating another Java class, is to simply copy and paste from an existing one. In the project navigator, click on your existing servlet file FetchNewsServlet.java, Ctrl/Cmd-C (copy), then paste using Ctrl/Cmd-V. 
  2. When the 'name conflict' dialog appears, enter the new name 'TokenServlet'. (Note: Don't append '.java')
    Alternatively, you would simply create an empy Java class 'TokenServlet' under the same package, 'com'.
  3. You will now replace the contents of this new servlet class with the following:

    package com;

    import java.io.IOException;

    import java.util.Date;
    import java.util.Random;

    import
    javax.jdo.PersistenceManager;
    import javax.servlet.http.*;


    import
    com.google.appengine.api.channel.ChannelService;
    import com.google.appengine.api.channel.ChannelServiceFactory;

    import
    com.model.ChannelClient;
    import com.model.PMF;

    @SuppressWarnings("serial")
    public class TokenServlet extends HttpServlet {
      public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
        resp.setContentType("text/plain");

        ChannelService channelService = ChannelServiceFactory.getChannelService();
        
        // Use random number for client id
        Random randomGenerator = new Random();
        String clientid = Integer.toString(randomGenerator.nextInt(100000));
        String token = channelService.createChannel(clientid);
        persistId(clientid);
        resp.getWriter().println(token);
      }

      private void persistId(String clientid) {

        ChannelClient client = new ChannelClient();
        client.setClientId(clientid);
        client.setTimestamp(new Date());
        PersistenceManager pm = PMF.get().getPersistenceManager();
        try {
          pm.makePersistent(client);
        } finally {
          pm.close();
        }
      }
    }


    As you may notice in the code, the doGet() responds to an HTTP GET request and creates a 'token' using the call:
    String token = channelService.createChannel(clientid);

    Notice that the clientid argument passed is simply a random number and the response is a token String value.

  4. At this point you may notice that this Java class does not compile cleanly. This is because we need to create the Java class ChannelClient. that will save this client id in the datastore. Later when broadcasting data, we will be querying clientids in order to broadcast to them.

  5. In the project navigator, locate the previous model class com.model/NewsItem.java. In a similar fashion, either copy and paste the NewsItem class, or simply create a new empty Java class with the name ChannelClient.java. Note: Make sure that it resides in the package com.model.

  6. Once the new Java class (ChannelClient) is created, you will replace the entire source of the class with:
    package com.model;

    import java.util.Date;
    import javax.jdo.annotations.IdGeneratorStrategy;
    import javax.jdo.annotations.PersistenceCapable;
    import javax.jdo.annotations.Persistent;
    import javax.jdo.annotations.PrimaryKey;

    import
    com.google.appengine.api.datastore.Key;

    @PersistenceCapable
    public class ChannelClient {
      @PrimaryKey
      @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
      private Key key;

      private String clientId;
      
      private Date timestamp;
      
      public String getClientId() {
        return clientId;
      }
      public void setClientId(String clientId) {
        this.clientId = clientId;
      }
      public void setTimestamp(Date timestamp) {
        this.timestamp = timestamp;
      }
      public Date getTimestamp() {
        return timestamp;
      }
      public Key getKey() {
        return key;
      }
      public void setKey(Key key) {
        this.key = key;
      }
    }



  7. Save all, and verify a clean compile. Your TokenServlet.java file should now compile cleanly also.
  8. To enable the servlet to respond to an HTTP GET request to /getToken, we must add the following to the war/WEB-INF/web.xml file:

    <servlet>
      <servlet-name>TokenServlet</servlet-name>
      <servlet-class>com.TokenServlet</servlet-class>
    </servlet>

    <
    servlet-mapping>
      <servlet-name>TokenServlet</servlet-name>
        <url-pattern>/getToken</url-pattern>
    </servlet-mapping>



  9. Save all and verify a clean compile.

    At this point we now have a servlet that will accept requests to the URL /getToken and respond with a token. This token can then be used by a JavaScript client using the JavaScript Channel API in order to open a secure communication channel to the server. 
    You'll also notice that as a secure connection is made, a clientid is persisted in the datastore. This will allow the server to broadcast to a registry of clients any messages.

Creating a JavaScript client to the Channel API
  1. For this step we will create an HTML file, newsclient.html which will contain the both a simple UI to display news feed data, as well as JavaScript code to attach to the server.
  2. In the project navigator, locate the war directory and right-click and select New->File.
  3. Enter the file name 'newsclient.html' and ensure that the file is being created in the war subdirectory.
  4. Once created, the editor will open the blank file. You can now add the following code.

     <html>
      <head>
       <script type="text/javascript" src="/_ah/channel/jsapi"></script>   
        <meta http-equiv="content-type" content="text/html; charset=UTF-8">
        <title>Channel API News Client</title>

        <script type="text/javascript">
        function onOpened() {
          connected = true;
          alert('connected to server'); 
        };

        function onMessage(m) {
          newMessage = JSON.parse(m.data);
          var newslink = '<a href=\"' + newMessage.link + '\" >' + newMessage.title + '</a>';
          var description = newMessage.description;

          document.getElementById('headlines').innerHTML = newslink;
          document.getElementById('description').innerHTML = description;        
        };

       function getToken(){
          var xhr = new XMLHttpRequest();
          xhr.open('GET', '/getToken', false);
          xhr.send(null);        
          if (xhr.status == 200) {
           return(xhr.responseText);
          }     
       };

      function openNewsChannel(token){
         var channel = new goog.appengine.Channel(token);
         var handler = {
              'onopen': onOpened,
              'onmessage': onMessage,
              'onerror': function() {},
              'onclose': function() {}
           };
         var socket = channel.open(handler);
       };

       function initialize() {
         var token = getToken();
         if ( token != null  && token != 'error') {
            //strip newline from returned token
             var cleantoken =  token.replace("\n", "", "g");
             openNewsChannel(cleantoken);  
           } else {
             alert('Error fetching token');
           }
       };
        </script>    
      </head>

      <body onload="initialize();">
       <h2>Breaking News Stories</h2>
       <hr/>
       <h3><div id="headlines">Waiting for headlines stream...</div></h3>
       <p><div id="description">Waiting for description...</div></p>   

       <hr/>
      </body>
    </html>


    As you may notice, this HTML page is using the Channel API JavaScript portion, which is linked in via "/_ah/channel/jsapi". As the page loads, the getToken() function is called during the initialization.  The function's purpose is to get a Channel API token from which to pass as an argument to open up a secure connection to the server. It does this by using the XMLHttpRequest object to issue an HTTP GET to the server. When the response comes back with the token, it uses that token to open up a communication channel to the server. When the connection is established, the onOpened() function is called asynchronously and a popup alert is displayed to the user indicating a successful connection.

    Later when messages are sent down the client, the onMessage() function will be called and the function will parse the incoming JSON data and use it to update the page dynamically (in this example with News feed data).

  5. Now that we've created the client page, we will update the web.xml file's welcome-file-list entry so that any request to the URL "/" will be directed to the newsclient.html page.
    Edit the welcome-file-list entry in the web.xml file as:
    <welcome-file-list>
      <welcome-file>newsclient.html</welcome-file>
    </welcome-file-list>


  6. Save all, and verify a clean compile for all files.

  7. Now you can run the app again, and this time you'll be able to access the newsclient.html and verify a secure connection to the server.
    Stop and restart the Web app as before and access the app at: http://localhost:8888/.

  8. Upon viewing the page, you will see the page successfully connect and a pop window will appear.


  9. Our final step is to write the code that will push news messages out to connected clients.

Using the Channel API - Pushing messages out to connected clients

This final task is to build the code necessary to push out messages to any connected clients. In short this code will loop through the persisted news items, and broadcast each news story to every connected client. In order to prevent the server from immediately pushing out message updates as fast as possible, we will be using deferred tasks. With deferred tasks, we will be able to create multiple tasks each with a delay, so they will execute at successively later times so the end user will see a delay giving them enough time to read the contents of the news item before it changes.

Creating a servlet to kick off the deferred tasks

Launching deferred tasks can be done in many ways, but for this task, we will create a servlet accessible from the URL /startMsgs. Once the servlet  is requested, it will loop through all of the NewsItem objects fetched from the datastore and then created a deferred task with a successively longer delay that will then broadcast this NewsItem message to every connected client.
 
  1. As before, either copy an existing Java class or create a new class (File->New->Class). The servlet will be named 'MessageServlet' and with the same package com.model.

  2. Once you've created a new MessageServlet.java file you will replace the entire content with the following:

    package com;

    import
    java.io.IOException;
    import java.util.List;
    import java.util.logging.Logger;

    import
    javax.jdo.PersistenceManager;
    import javax.servlet.http.*;

    import
    com.google.appengine.api.taskqueue.QueueFactory;
    import com.google.appengine.api.taskqueue.TaskAlreadyExistsException;
    import com.google.appengine.api.taskqueue.TaskOptions;
    import com.model.NewsItem;
    import com.model.PMF;

    @SuppressWarnings("serial")
    public class MessageServlet extends HttpServlet {
      
      private static final Logger log = Logger.getLogger(MessageServlet.class.getName());
      public static final long DELAY = 3000;
      
      public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
        resp.setContentType("text/plain");
        resp.getWriter().print("starting messages");

        PersistenceManager pm = PMF.get().getPersistenceManager();

        
    // Get all current news Items
        String query = "select from " + NewsItem.class.getName();
        List<NewsItem> newsitems = (List<NewsItem>) pm.newQuery(query).execute();
        
        // Loop through current news items and broadcast to clients
        int i=0;
        for (NewsItem ni : newsitems) {
          i++;
          if (ni.getTitle() != null && ni.getLink() != null ){
            String description = null;
            if (ni.getDescription() != null){
              description =  ni.getDescription().getValue();
            }
            
            TaskOptions taskOptions = TaskOptions.Builder.withDefaults()
                .countdownMillis(DELAY*i).payload(new BroadcastToClientsTask(ni.getTitle(), ni.getLink(), description));
       
            try {
                QueueFactory.getDefaultQueue().add(taskOptions);
            } catch (TaskAlreadyExistsException e) {
              e.printStackTrace();
            }        
          }
        }
        pm.close();
      }  
    }


    Upon inspection of this code, we see that it first queries all the
    NewsItem datastore entities, and then loops through and launches a deferred task, BroadcastToClients which does the actual broadcast to the clients.

    You'll notice that currently the
    MessageServlet.java doesn't compile yet, because we need to create our BroadcastToClients class.

  3. As before, create a new Java class named BroadcastToClients with the same package com. Then place the following code in the class:
    package com;

    import
    java.util.List;
    import java.util.logging.Logger;

    import javax.jdo.PersistenceManager;

    import com.google.appengine.api.channel.ChannelFailureException;
    import com.google.appengine.api.channel.ChannelMessage;
    import com.google.appengine.api.channel.ChannelService;
    import com.google.appengine.api.channel.ChannelServiceFactory;
    import com.google.appengine.api.taskqueue.DeferredTask;
    import com.google.appengine.repackaged.org.json.JSONException;
    import com.google.appengine.repackaged.org.json.JSONObject;

    import com.model.ChannelClient;
    import com.model.PMF;

    public
    class BroadcastToClientsTask implements DeferredTask {

      private static final Logger log = Logger.getLogger(BroadcastToClientsTask.class.getName());

      private String title;
      private String link;
      private String description;

      public String getTitle() {
        return title;
      }

      public void setTitle(String title) {
        this.title = title;
      }

      public String getLink() {
        return link;
      }

      public void setLink(String link) {
        this.link = link;
      }

      public String getDescription() {
        return description;
      }

      public void setDescription(String description) {
        this.description = description;
      }

      BroadcastToClientsTask(String title, String link, String description) {
        this.title = title;
        this.link = link;
        if (description != null) {
          this.description = description;
        }
      }

      public void run() {
        if (this.getTitle() != null && this.getLink() != null) {
          broadcastToClients(this.getTitle(), this.getLink(), this.getDescription());
          log.info("Broadcasted news message to clients");
        }
      }

      private void broadcastToClients(String title, String link, String description) {
        JSONObject jsonMessage = null;

        
    // Get all channel client ids available
        String query = "select from " + ChannelClient.class.getName();
        PersistenceManager pm = PMF.get().getPersistenceManager();
        List<ChannelClient> ids = (List<ChannelClient>) pm.newQuery(query).execute();

        ChannelService channelService = ChannelServiceFactory.getChannelService();

        for (ChannelClient m : ids) {
          String client = m.getClientId();
          try {
            jsonMessage = new JSONObject();
            jsonMessage.put("title", title);
            jsonMessage.put("link", link);
            jsonMessage.put("description", description);

            System.out.println("sending json stream: " + jsonMessage.toString());
            System.out.println("to client: " + client);

            channelService.sendMessage(new ChannelMessage(client, jsonMessage.toString()));

          } catch (JSONException e) {
            e.printStackTrace();
          } catch (ChannelFailureException e) {
            e.printStackTrace();
          }
        }
        pm.close();
      }
    }


    Upon inspection of this code, we first see that it implements the DeferredTask interface. It also has a constructor that assigns the values for title, link and description upon creation.  As the task is executed, it's
    run() method is called which then calls the method, broadcastToClients() which accepts the three String arguments for title, link and description and then constructs a JSON message with the data.
    It then fetches all of the current ChannelClient entities from the datastore and loops through and broadcasts the messsage to each client using:
    channelService.sendMessage(new ChannelMessage(client, jsonMessage.toString()));
     
  4. Before we can run the app and begin broadcasting news items to clients, we must enable the MessagesServlet by adding an entry in the web.xml.

    <servlet>
      <servlet-name>MessageServlet</servlet-name>
      <servlet-class>com.MessageServlet</servlet-class>
    </servlet>

    <
    servlet-mapping>
      <servlet-name>MessageServlet</servlet-name>
      <url-pattern>/startMsgs</url-pattern>
    </servlet-mapping>


  5. Save all and verify a clean compile.
  6. Now restart the application and the following to test the full application:
    1. Before accessing via the news client page, you must make sure you have NewsItems entities by manually kicking off the cron job to fetch the news items: http://localhost:8888/cron/fetchNews 
    2. Once a fresh batch of news items have been fetched the main client page can be accessed at:
      http://localhost:8888 
      This will open a client connection. 
      Once a successful connection has been achieved, you can then start the message broadcasting.
    3. Open a new browser window, and turn on the message broadcast by accessing the app at:
      http://localhost:8888/startMsgs
    4. Once the messages start, you'll see "starting messages".  Then return to your first browser window to watch the messages come streaming in.


      Note: because the first Google news feed only has around 10 entries, you will have to relaunch the messages via /startMsgs URL to see it run again as it concludes fairly quickly.
  7. Feel free to repeat the same exercise over again, but with multiple browser client windows open to see simultaneous updates to multiple clients.

Congratulations, you have completed the core portion of the Channel API codelab!

Making the Channel API app even better - Cleaning old Channel API Clients

As you may have noticed, multiple clients accessing the app will build up a list of clientids needlessly, and since the client ids on the server are only  kept live for 2 hours, another cron job could be created to periodically clean the ChannelClient entities.

This is easily done by creating the servlet, CleanClientIdsServlet, and having it run every 2 hours.
  1. Create a new class, CleanClientIdsServlet with the same package, com. Then add the following contents into the class.

    package com;

    import java.io.IOException;
    import java.util.Date;
    import java.util.List;
    import java.util.logging.Logger;

    import
    javax.jdo.PersistenceManager;
    import javax.servlet.http.*;

    import
    com.model.ChannelClient;
    import com.model.PMF;

    @SuppressWarnings("serial")
    public class CleanClientIdsServlet extends HttpServlet {
      public static final int MINUTE = 3600;
      public static final int MILLI = 1000;

      private static final Logger log = Logger.getLogger(CleanClientIdsServlet.class.getName());

      public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
        resp.setContentType("text/html");
        resp.getWriter().println("Cleaning Clients...");
        
        // Get all channel client ids available
        String query = "select from " + ChannelClient.class.getName();
        PersistenceManager pm = PMF.get().getPersistenceManager();
        List<ChannelClient> ids = (List<ChannelClient>) pm.newQuery(query).execute();

        
    // Delete any clientids with timestamps older than an hour
        Date oneHourAgo = new Date();
        oneHourAgo.setTime(oneHourAgo.getTime() - MINUTE * MILLI);    

        for (ChannelClient c : ids) {
          Date clientDate = c.getTimestamp();
          
          // if client timestamp is older than 1 hour, delete it.
          if (c.getTimestamp().before(oneHourAgo)){
            // Delete this client
            log.info("deleting stale client: " + c.getClientId());
            pm.deletePersistent(c);
          }
        }
      }
    }



  2. Add the following servlet entry to the web.xml file.
    <servlet>
      <servlet-name>CleanClients</servlet-name>
      <servlet-class>com.CleanClientIdsServlet</servlet-class>
    </servlet>

    <
    servlet-mapping>
      <servlet-name>CleanClients</servlet-name>
      <url-pattern>/cron/cleanClients</url-pattern>
    </servlet-mapping>


  3. And to register the cron job, enter the following in the cron.xml file.
    <cron>
      <url>/cron/cleanClients</url>
      <description>Clean out stale client ids every 2 hours</description>
      <schedule>every 2 hours</schedule
    >
    </cron>


  4. At this point you can redeploy the app to the cloud and run it again from the cloud entirely. To do this, remember after deployment to go ahead and fetch a fresh set of news items using '/cron/fetchNews' before executing '/startMsgs'.

Congratulations! You have completed part 3 of the App Engine Java codelab where you used a combination of the following App Engine services:
  • URLFetch 
  • Cron
  • Task Queues (deferred)
  • Channel API
  • Datastore
As a supplement to this lab, you may also download a complete version of this application in an Eclipse Workspace. The following steps demonstrate how to download, install and run the complete sample application built in this codelab.

Installing and running the complete Channel API sample Application

Installing and running the complete application can be done in order to see the entire set of services in this section working together.  These steps can be done even before doing the individual steps to build this application.
  1. Download the sample application from here.
  2. Unzip this file into your Eclipse workspace.
  3. In Eclipse select File->Import  and select "General->Existing Projects into Workspace"
  4. Select the root directory of the unzipped document. (NewsClientChannelAPI) and click "Finish".
  5. Verify that the project imported into Eclipse and successfully compiles.
Running the app. 
(Important, upon first run, you must first seed the News feed database.)
  1. In Eclipse, right-click the project NewClientChannelAPI and select "Run as -> Web Application"
  2. In the console, you should that http://localhost:8888 is now accessible.
    • INFO: The server is running at http://localhost:8888/
  3. Important before accessing the application at the root directory, you must first seed the datastore with news feeds. This is done by executing the following "cron" job manually
    1. View http://localhost:8888 in a browser.
    2. After connecting to the server, manually launch the cron job to fetch news feed data by accessing: http://localhost:8888/cron/fetchNews
    3. This will fetch the current news feed data and persist it into the local datastore for further use. You should also see the data fetched displayed in the browser window.
    4. Optional: Go to the local SDK Console and verify that the data has been persisted.
  4. Now that the local datastore has been populated with new data, we can now access it from the application. Return to the URL: http://localhost:8888
  5. This will open the page: newsclient.html
  6. You should see the page "Breaking News Stories" as a header. You should also see browser Alert popup indicating that you are now  'connected to the server'. - Click ok.
  7. Now you can launch the Task that will start broadcasting out a series of news headlines to this browser. To launch this task, open a new tab and access the url: http://localhost:8888/startMsgs
    This will launch a task queue that will start broadcasting messages to any clients that are currently connected.
  8. Now return to the new client page (newsclient.html) in the other tab.
  9. You should now see news headlines and updates appearing in the browser window every 3 seconds!
  10. You can also return to the Eclipse console and watch as the messages are broadcast out.