2. Requesting Graded Feedback

You can request to view feedback on your latest submission by creating a comment with @310-bot #<deliverable_id> on a commit (see figure below for an example for the c0 deliverable). You may request feedback twice per calendar day--within minutes of each other, or at 10am and 10pm, or at any other two times per day. The choice is yours! Beware: There is no way to cancel a request, so make the requests carefully. A project that fails to build on AutoTest will NOT consume one of these requests.

AutoTest feedback is meant to help you gauge your progress and to give gentle hints if you are stuck. It is not meant as a replacement for good software engineering practices like specification analysis or test suite strengthening, nor is it meant to replace Piazza or office hours!

Note that feedback may take more than 12 hours to be returned, especially close to the deadline when AutoTest is under heavy load. If you wish to make the most of this feedback, we recommend you start earlier (when it will likely be returned within seconds!). AutoTest runs pre-emptively in the background, so when load is low AutoTest may be able to return your result immedeately. If request an already-run result the cached version will be returned, AutoTest will not run again (you will only be 'charged' once for a given <deliverable, SHA> tuple).

Your code can only be graded (and feedback can only be given) if your code builds (which you can verify with yarn build), passes prettier (which you can verify with yarn prettier:check), and passes lint (which you can verify with yarn lint:check). Obviously non-building code cannot run, but trying to disable prettier or lint (either through configuration files or in-code directives) can cause your code not to run. Please don't try to do this! If this happens to you though, AutoTest will be clear that the code did not run, so fixing the problem should be relatively straightforward.

3. Requesting Test Suite Feedback

Interpreting Test Suite Feedback

Here are a few types of feedback that you will observe and an explanation of what each one means:

Skipped tests

Skipped tests are usually caused by an error being thrown while running your tests against our implementation (i.e. a bug in your code). It could be an ill-formatted JSON for the query tests, zip files required by tests that aren't committed to git, incorrect file paths in tests, etc.

Your tests failed against a reference implementation

There is a bug in your test. Tests can have bugs, just like implementation code!  A test has a Setup, Execution and Validation. Now your job is to figure out which one of these is causing the issue. 

• The Setup contains any code required to get your application into the desired testable state.

• The Execution contains your method under tests. 

• The Validation contains any asserts. 

For example:

it("should list one dataset", async function() {

// Setup
  await insightFacade.addDataset("ubc", content, InsightDatasetKind.Sections);

  // Execution
  const datasets = await insightFacade.listDatasets();

  // Validation
  expect(datasets).to.deep.equal([{
    id: "ubc",
    kind: InsightDatasetKind.Sections
    numRows: 64612

]);
})

Setup

• Check each parameter, is it correct? Am I passing in a valid value for each parameter?

• Am I doing all the necessary setup for this test? If I'm testing performQuery, have I added the dataset to query?

• Am I handling promises correctly?

Execution

•  Check each parameter, is it correct? Am I passing in a valid value for each parameter?

• Am I handling promises correctly?

Validation

This is the only thing you can test locally!  First, update your InsightFacade implementation to return the expected result whether it be a promise resolving with a value or a promise rejecting with an error.

For example, below I've updated listDataset to return the expected result for the above test:

listDatasets(): Promise<InsightDataset[]> {

   return Promise.resolve([{id: "ubc", kind: InsightDatasetKind.Sections, numRows: 64612}]);

}

After updating your implementation, your test should pass locally.

Timeout

AutoTest will timeout if your tests are taking too long to run. A common issue is repeating expensive work that only needs to be done once. For example, adding a dataset is expensive! For performQuery tests, we recommend adding your datasets once in a before rather than repeatedly in beforeEach.

In addition, avoid adding the given dataset (pair.zip) when you don't have to. You can create a smaller dataset with only a couple valid sections. This will suffice for the vast majority of your tests.

Unhandled promises

You have an unhandled promise in your tests! You will need to go through each test and ensure you are handling promises correctly. If you are unsure what it means to "handle" a promise, checkout the async cookbook module. 

Missing files

Missing files are a common source of difficulty, most often caused by files missing from version control, referencing files in a case-insensitive manner, or accessing directories that exist in dev but not in prod (or on a partner's machine). Emitting any form of `ENOENT` error message to the console could trigger these warnings, so be sure to log your file accesses carefully and avoid re-throwing Node's file-related errors or just printing them straight to the console.

Flaky tests

This message occurs when a test passes on one run, but fails on another. Most likely, the flakyness is caused by the above issue - Unhandled Promises. As a first step, we recommend you to review your test suite for any unhandled promises.

[Dynamic test name omitted]

You will see this error if you have a dynamic test name in your test suite. The names of the failing tests will only be shown if the exact string of the test name is present in your source files.

This means a test written like
it("Should" + " add a valid dataset", function () {...});
will not have its name reported back upon failure, as the test name is created dynamically. (This restriction is in place to prevent grading data exfiltration.)