Performance is a feature at Khan Academy. Focusing on both frontend (see Best practices for speeding up your website) and backend perf is a priority.
Google briefly describes a good set of practices for App Engine performance in Best practices for writing scalable applications. It's a five-part series, the most useful of which are:
To get started using Google's BigQuery, see the instructions in the BigQuery Cookbook.
To experiment on logs-based queries, it can be handy to create a temporary table with the subset of interesting rows. For example, when analyzing out-of-memory errors, I created a table containing only logs with out-of-memory issues. This table was a few MB in contrast to the original tens of GBs of data: