You will also learn simple meta-programming, i.e., writing a program that modifies an existing program. You will use Tampermonkey to modify a web application loaded into your browser.
You will combine d3 and Tampermonkey to place Facebook users on a map. This experience will teach you how you can easily do other cool things with web pages.
For this homework, you will work alone. You can consult d3 and other technologies with your class mates but you need to implement your own solution.
Monday Jan 28, 2013, 11:59 PM
Have you ever wondered where your friends live across the world? Could you use Facebook to visualize their locations? Similarly, could you visualize the geographical distribution of Facebook users commenting on a controversial post?
You probably know that many Facebook users reveal where they live. We will write a script that collects this information and draws their locations on the world map, which it will insert on the top of the Facebook page:
We recommend that you develop the solution in three consecutive steps, described below. You will not submit the intermediate solutions but having solutions to Steps 1 and 2 in hand will make the last step easy because you will simply integrate the two.
We will be testing your homework projects using Chrome 24 (not Chromium). Since different versions of Chrome behave differently, make sure you are using Chrome Version 24 to avoid any compatibility issues.
In this step, you are asked to use d3 to render a world map and draw Berkeley on it. In addition to d3, you will learn how to use a geocoding API, which translates a location name to its coordinates. In this program, "Berkeley, CA" will be hard-coded in your program; in Part 3, you will replace this string constant with locations scraped from Facebook pages.
STEP 1 Read a tutorial
It does not hurt to read the whole tutorial. Paying special attention to drawing bar charts may pay off when you need to create simple visualizations for debugging of your compiler in later cs164 projects.
STEP 2 Test yourself
Before you proceed, test your understanding of d3 data binding with this exercise. Specifically, we want to help you understand the semantics of
STEP 3 Create index.html and start a local web server
Create index.html in your working directory. Add some html content to the file.
References that you may need: What is HTML?
Chrome enforces strict permissions for reading file out of the local file system. Therefore, in Part I, you will need to run your own local sever. In the directory where you created your index.html, start your own local web server by running Python's built-in server:
Once the server is running, direct your browser to http://localhost:8888/. The browser will display your index.html file.
STEP 4 Draw the world map
First, we need world map data (shapes of countries and such). You can use the TopoJSON data at http://www.cs.berkeley.edu/~bodik/world.json or you can download this file to your working directory so that your server can serve it for you as http://localhost:8888/world.json.
Next, in index.html, use d3 to draw the world map.
STEP 5 Draw Berkeley on the map
Use Google Geocoding to find latitude and longitude of Berkeley. We recommend that you use the JSON output format. Next, convert latitude and longitude to pixel coordinates on the map, using the D3 projection object. Finally, draw a red circle on these pixel coordinates.
STEP 6 Fix the asynchrony bug
Our early prototype had a bug that manifested itself as the non-deterministic behavior shown in these screen shots. You may have the same bug. To try to reproduce the bug, refresh the browser, rerunning the program, and observe whether the circle is sometimes on top of the map, and sometimes under the map.
Berkeley over the map Berkeley under the map
What causes this behavior? Can you rewrite the program to ensure that this behavior does not happen in your program no matter how the servers answer your requests?
Hint: read the d3 tutorial on the lack of z-indices in SVG.
In more detail, we are interested in locations of users involved in Facebook activities such as postings on walls, picture posts, comments and likes on posts, etc. To extract locations, you follow to the user pages by clicking on their names; when on their pages, you can see the text "Lives in ..." if they share the information.
STEP 1 Work with Tampermonkey
What to do? Install Tampermonkey in your browser and familiarize yourself with Tampermonkey configuration.
Additional recommendation: learn browser debugging tools
You will use Chrome Developer Tools often so make them your friend. Chrome Developer Tool comes with your Chrome browser. The most basic use is printing messages to Developer Tools' console, with the following command, which you can insert into your Tampermonkey script:
STEP 2 Extract links to friends' pages
The web page is represented by a DOM tree. The links to your friends' pages can be easily extracted in a Tampermonkey script by selecting suitable DOM elements. You will do so by writing a DOM query. You can use d3 methods to write this selection query. This tutorial might be useful.
Use Chrome Developer Tools to help your figure out what DOM queries are needed to find the set of DOM nodes that contain the links.
Hint: Link to your friends' page using the Inspect Element command of the Chrome debugger.
Notice that the DOM node has "actorDescription actorName" class. You should use this class information to obtain the link to the poster's page. You will need to do something similar for commenters, likers, etc.
STEP 3 Obtain the content of friends' pages
Invoke XMLHttpRequest to query data from the link you found in the previous step.
Hint: You might want to set
STEP 4 Extract location where your friend lives
STEP 5 Print the location to the developer tool's console.
Verify that the string only contains the city, state, and/or country. It should not contain any irrelevant string, such as a fragment of HTML. Using
Now, we are ready to put everything together by feeding output of Part II into Part I.
STEP 1 First, you will insert the svg element with the map as the first item in the body of Facebook News Feed page, so that it appears visually on top of the page. You can use d3 methods to modify the Facebook DOM.
Note: Since you are running Tampermonkey script in the facebook.com domain, you will not be able to access the world map data from you local file system; you need to query the data directly from http://www.cs.berkeley.edu/~bodik/world.json. The methods
STEP 2 After successfully drawing the map on Facebook, you will extract as many friends' pages as possible from your news feed. Find out where they live and draw their locations on the map!
Here you may run into a problem that when your Tampermonkey script is executed, the DOM does not yet contain the names of your friends. This is because the facebook JS scripts load these dynamically, and often take their sweet time, even if it may seem to you that the page content loads instantaneously.
Hint: Registering a DOM event handler
Often we want something to happen after a specific event occurs. The event could be a mouse click, mouse hover or a particular DOM element being modified. This is done through event handlers. Every element in the DOM has a set of predefined events. One can add a handler, i.e., a function that will be executed after a specific event, using the
Links to useful sites on this topic can be found at:
Hint: Use a timing events to run your script after the DOM has been completed
STEP 3: Solving remaining challenges
Once we have your script working. Let’s see how you can make your script better! This part is mandatory.
For this project you will turn in the following:
Write two short paragraphs, each up to 70 words. Yes, that's very little text, so start with a longer draft and then revise it to communicate only your best thoughts and so concisely.
1st paragraph: Where did your time go? Reflect on your experience with developing Facebook Map app and identify the "developer time bottleneck".
2nd paragraph: Suggest how to remove the frustration or accelerate the identified bottleneck. Propose a tool, a development methodology, a library, a language extension, or some other innovation that would have simplified your programmer task.
Screen recordings can be made with the following software, depending on what system you are using.
OS X: You can use a free trial version of iShowU HD or create a screen recording with Quicktime X.
Windows: You can use the free Cam Studio.
Linux: You can used recordMyDesktop, but You might have to convert the video file afterwards.
Using your instructional account, submit the following three files: