Udacity Project Assistant

Overview

The Project Assistant is a system that students interact with to get automatic feedback and test their project code before submitting for review. The system includes:

• A client-side CLI tool that students run to submit their code
• A web backend that handles submission requests
• A group of remote sandbox servers that executes a series of scripts to test each submission.

Students typically only have direct access to the CLI tool.

Submitting a project with the CLI tool starts a chain of events:

1. The CLI tool makes an https session connected to the web backend
2. The project files (specified in the .udacity-pa folder of the project git repo) are zipped (locally on the student's machine) and sent to the server
3. The zip file is saved and sent to one of the sandboxes, which tests the submission by:
   1. Cloning a git repository containing custom test code (typically written by Udacity content developers) for the project being tested
   2. Downloading the submitted zip file and unzipping the contents into a temporary folder
   3. Executing the test code (which has access to the submitted code)
   4. Sending feedback to the Udacity reviews database and back to the student on the open connection via the CLI
4. The CLI tool renames the zip file to include the submission ID number from the database and saves the test feedback locally as a JSON file

NOTES:

• The sandboxes have OS-level resource constraints enforced (typically execution time, number of open files, storage space, RAM limits, etc.) that will cause the sandboxes to abort if student code exceeds the limits.
• Some Udacity projects also require manual review to complete; these projects are not completed until students submit the zip file created by the CLI through the classroom for manual review.

Installing the Project Assistant CLI Tool

The Project Assistant CLI tool can be installed from PyPI via pip:

pip install udacity-pa

Submitting Projects with the Project Assistant CLI

Once the tool is installed, you can submit most projects by changing your working directory to a valid Udacity project folder (one that contains a .udacity-pa folder -- example), and invoking the "submit" procedure:

udacity submit

See the Project Assistant FAQ and the common errors pages if you encounter any problems.

Pretty-Printing JSON Feedback

The CLI automatically pretty-prints the feedback generated for each submission to the terminal when the connection closes. However, many students run the tool in a window that closes with the network connection, or otherwise want to regenerate the feedback outside the PA. There are a couple of easy ways (and others exist). JQ is especially nice because it colors the json (you can use the pygments library to enhance the python versions).

Note: replace "foo.json" in the commands below with the name of your specific feedback file.

If you have the cat Bash utility:

1. Use Python json.tool module

cat foo.json | python -m json.tool

2. Install & use jq

Install the jq tool, then run:

cat foo.json | jq .

3. Use Python (compact lines)

cat foo.json | python -c "import json; print(json.dumps(json.loads(input()), sort_keys=True, indent=2))"

If you do not have the cat utility:

1. Use Python (no dependencies)

python -c "import sys; import json; print(json.dumps(json.load(open(sys.argv[-1], 'r')), sort_keys=True, indent=2)" foo.json