Search this site
Embedded Files
Pathway Pre-Check Improvement
  • Home
  • Problem
    • Background Research
    • Previous Solutions
  • Prototype Design
    • Design Criteria
    • Brainstorming
    • Plan
    • Prototype Creation
  • Final Prototype
    • Data and Testing
      • Pilot Tests
      • Test #1
      • Test #2
    • Improvements
    • Impacts
    • Conclusion
  • Future Implications
    • Continue My Project?
  • Reflection
  • Extras
    • Rubric Explanation
    • Presentations
Pathway Pre-Check Improvement
  • Home
  • Problem
    • Background Research
    • Previous Solutions
  • Prototype Design
    • Design Criteria
    • Brainstorming
    • Plan
    • Prototype Creation
  • Final Prototype
    • Data and Testing
      • Pilot Tests
      • Test #1
      • Test #2
    • Improvements
    • Impacts
    • Conclusion
  • Future Implications
    • Continue My Project?
  • Reflection
  • Extras
    • Rubric Explanation
    • Presentations
  • More
    • Home
    • Problem
      • Background Research
      • Previous Solutions
    • Prototype Design
      • Design Criteria
      • Brainstorming
      • Plan
      • Prototype Creation
    • Final Prototype
      • Data and Testing
        • Pilot Tests
        • Test #1
        • Test #2
      • Improvements
      • Impacts
      • Conclusion
    • Future Implications
      • Continue My Project?
    • Reflection
    • Extras
      • Rubric Explanation
      • Presentations

Test #2

Small Group

Responders & Data Integrity

For this second test, my goal was, again, to obtain statistics relating to the first part of my design criteria. The only difference this time was I tested more people. Essentially, I tested a small group of students and teachers (48 total responders) and asked them a couple of questions relating to the number of clicks to complete each method, the number of highlights to complete each method, and how long it took them to complete each method while filling out a randomly selected academic planner. 

Responder Groups of People Chart:

The overall percent difference from this data, as shown in the picture on the right, from students, was about 29%.

Looking at the picture on the left, it's clear that the overall spread of data was very spread out among staff and students. I then needed to calculate a percent difference for this data compared with all of GMHS.

In terms of percent difference from the teacher side of the respondents, there isn't one, as I only added teachers into the mix, to give me a bigger variety of users. With a larger variety of users, it would help me user-proof the website for future users as your age/technical know-how wouldn't hinder your ability to complete this new website method

Random Academic Planner Groups Chart:

Now, since each person who completed test #2 was given a random academic planner to use, I also calculated a percent difference for the overall spread of 9th, 10th, 11th, and 12th, grade planners used. This time, the percent difference was 14.06%.

Calculated Overall Percent Difference: 21.5%

Now, considering that this survey was completely optional, and not everyone had to participate in each class that I tested, I would argue that a percent difference of just above 20% is pretty good for this test.

Results...

Overall Changes From All Grades

61.25% net decrease in total clicks (Criteria Met)

100% net decrease in total highlights (Criteria Met)

20.48% net decrease in total time (Criteria Not Met)

Margin of Error

The margin of error in this set of data is tough to decipher. I say this because all of the data inputted by one person could have been vastly different than what someone else inputted. This is because I didn't specify a margin by which people needed to enter their data. This theoretically wouldn't be an issue, however, since everyone who completed this test was supposed to complete both the document method and the website method for their given transcript. Then, since they would complete both methods, their rounding methods would also be the same for both methods, reducing the issue of whether or not their inputs would skew my data in the end. Whether everyone still did both methods is still, unfortunately, left up to the tester. Long story short, the margin of error for the data could be anywhere from 1 click to hundreds of clicks, or 1 second to a couple minutes. If I were to do this test again, I would specify a margin on which I want the testers to round their numbers to. 

Possible Expiramental Errors

Some possible experimental errors are:

  • Incorrectly counting the number of clicks while filling out either method of pre-check

  • Incorrectly stopping/starting stopwatch when beginning or ending each method of pre-check

  • Misunderstanding how to use either method of pre-check and taking additional time than what is needed

  • Incorrectly counting the number of highlights used in the document method of pre-check

  • Misunderstanding instructions in the survey and giving false data

  • Due to lack of motivation, not giving accurate data in survey response

  • Due to missing classes in the website database, not understanding to skip these classes, and instead giving up

  • The idea of random chance not being fully removed as a culprit when discussing my data and not keeping my "p-value" below the 0.05 margin.

Analysis

Even though I did manage to meet two of my criteria, when it comes to the timing of things, there were a couple of things that people still seemed to struggle with. These things were:

  • Understanding that you had to enter all classes from your high school career

  • Understanding that you had to enter both semesters of classes, as opposed to just one, and the other class will add itself.

  • Understanding what certain classes were named

  • Knowing that if you couldn't find a class just skip it

  • Knowing that the academic planner was your best friend for completing this website method of pre-check

  • How to effectively search for classes and use the search bar as a whole.

After completing this test, it seems clear to me that instructions were still the number one issue of my new method of pathway precheck. If I were to give better and more straightforward instructions or adjust the way people enter classes to a sectioned-off set-up, people might have been able to reach that 50% decrease in the total time mark.

Detailed Data, Before Overall

Calculation...

9th Grade Data

Document Method

Among the 8 responders who filled out the website method and tested for time, clicks, and highlights, the averages were...

  • 76.5 total clicks

  • 10.8 highlights

  • 9 minutes

  • & 2 of the 8 people gave up

Website Method

Among the 13 responders who filled out the website method and tested for time, clicks, and highlights, the averages were...

  • 55 total clicks

  • 0 highlights

  • 4.71 minutes

  • & No one gave up

9th - Overall

28.1% decrease in total clicks

100% decrease in total highlights

47.67% decrease in total time

9th - "2-Samp T-Tests" (Statistical Proof)

These tests were conducted a check step to ensure that my data was not caused due to random chance.

The results of this data show how the "p-value" for both clicks and total time is greater than 0.05. This, unfortunately, means that we can't statistically prove that this data was not skewed by random chance. This, however, does not invalidate my data, but is rather an experimental error.

10th Grade Data

Document Method

Among the 5 responders who filled out the website method and tested for time, clicks, and highlights, the averages were...

  • 76 total clicks

  • 17.5 highlights

  • 8.17 minutes

  • & 1 of the 5 people gave up

Website Method

Among the 9 responders who filled out the website method and tested for time, clicks, and highlights, the averages were...

  • 21.89 total clicks

  • 0 highlights

  • 5.55 minutes

  • & No one gave up

10th - Overall

71.2% decrease in total clicks

100% decrease in total highlights

32.07% decrease in total time

10th - "2-Samp T-Tests" (Statistical Proof)

These tests were conducted a check step to ensure that my data was not caused due to random chance.

The results of this data show how the "p-value" for both clicks and total time is greater than 0.05. This, unfortunately, means that we can't statistically prove that this data was not skewed by random chance. This, however, does not invalidate my data, but is rather an experimental error.

11th Grade Data

Document Method

Among the 5 responders who filled out the website method and tested for time, clicks, and highlights, the averages were...

  • 100 total clicks

  • 12.5 highlights

  • 7.52 minutes

  • & 1 of the 5 people gave up

Website Method

Among the 9 responders who filled out the website method and tested for time, clicks, and highlights, the averages were...

  • 33.78 total clicks

  • 0 highlights

  • 8.42 minutes

  • & No one gave up

11th - Overall

66.22% decrease in total clicks

100% decrease in total highlights

11.97% increase in total time

11th - "2-Samp T-Tests" (Statistical Proof)

These tests were conducted a check step to ensure that my data was not caused due to random chance.

The results of this data show how the "p-value" for both clicks and total time is greater than 0.05. This, unfortunately, means that we can't statistically prove that this data was not skewed by random chance. This, however, does not invalidate my data, but is rather an experimental error.

12th Grade Data

Document Method

Among the 7 responders who filled out the website method and tested for time, clicks, and highlights, the averages were...

  • 268.2 total clicks

  • 25.6 highlights

  • 12.3 minutes

  • & 2 of the 7 people gave up

Website Method

Among the 10 responders who filled out the website method and tested for time, clicks, and highlights, the averages were...

  • 55.1 total clicks

  • 0 highlights

  • 10.56 minutes

  • & No one gave up

12th - Overall

79.46% decrease in total clicks

100% decrease in total highlights

14.15% decrease in total time

12th - "2-Samp T-Tests" (Statistical Proof)

These tests were conducted a check step to ensure that my data was not caused due to random chance.

The results of this data show how the "p-value" for both clicks and total time is greater than 0.05. This, unfortunately, means that we can't statistically prove that this data was not skewed by random chance. This, however, does not invalidate my data, but is rather an experimental error.

Additional Feedback

At the end of my survey, I asked for any additional feedback that the user might have. The main things that were listed were:

  • Some honors/AP classes were missing

  • It was tedious to not be able to find classes by just typing in a few characters from each word in name of the class

  • It was a little confusing on what classes everyone was supposed to enter

    • High school career, vs. current year

  • Some classes didn't seem to count when user's clicked to the next page

  • Most people appear to like the idea of SHERPA and that with a little more refinement, it can be a very powerful tool for students at GMHS

Google Sites
Report abuse
Page details
Page updated
Google Sites
Report abuse