Test #1 With

Entire School

For this first pilot test, I tested the entire school through one RamTime period and obtained data through a Google Form. Specifically, I tested them on three scales: "user-friendliness", "comfortability", and "information-obtained" to help me understand if my solution is viable. I also asked an "any other feedback" question at the end of the survey where the participants were allowed to list other critiques for the website.

Responders & Data Integrity

This first test was rolled out through Ram Time at GMHS using a Google survey. This survey was completely optional for students to fill out meaning I didn't get all of the underclassmen to fill it out. Using the picture on the left, the spread of total responders can be seen. Overall, a very even spread of data. Seniors are not present here as the website and Google survey were strictly released to freshmen, sophomores, and juniors, in order to prepare them for the next year's class registration process.

The overall percent difference from this data, as shown in the picture on the right, from students, was about 35.1%. Now, because the test was released to about 819 students, and everyone had the option of not participating, getting a percent difference lower than 50% is not bad for this set of data collection.

Results...

Overall Changes From All Grades

(Above average category used for these percentages)

15.3% net increase in user experience (Criteria Met)

32.1% net increase in comfortability (Criteria Met)

8.1% net increase in information obtained (Criteria Met)

Margin of Error

The margin of error in this data is about 25%. This is due to the fact that students were required to select a ranking from 4 options, meaning that any values in-between: below average, average, above average, and extremely above average were not correctly answered for. Apart from those in between values, however, there was no other margin of error as the feedback question at the end of the survey was a short response answer.

Possible Expiramental Errors

Some possible experimental errors are:

Rounding a chosen answer to one of the four possible answers as there were no other options.
Misunderstanding how to use either method of pathway pre-check and therefore not reporting correct analysis
Misunderstanding instructions in the survey and giving false data
Due to lack of motivation, rushing, and not giving accurate data in survey response
Due to missing classes in the website database, not understanding to skip these classes, and instead giving up
Random chance not being eliminated as a factor with two of my tests

Analysis

Even though I did manage to meet all of my criteria for this test, when it comes to the website, there were a couple of things that people still seemed to struggle with. These things were:

Understanding that you had to enter all classes from your high school career
Understanding that you had to enter both semesters of classes, as opposed to thinking just one, and the other class will add itself.
Understanding what certain classes were named
Knowing that if you couldn't find a class just skip it
Knowing that the academic planner was your best friend for completing this website method of pre-check
How to effectively search for classes and use the search bar as a whole.

After completing this test, it seems clear to me that instructions were still the number one issue of my new method of pathway precheck. If I were to give better and more straightforward instructions or adjust the way people enter classes to a sectioned-off set-up, my increased margins for comfortability and information obtained likely would have increased.

Detailed Data...

User Friendliness

...the ability for the buttons, clicks, and other javascript/php features of the website to work correctly. This is not focused on whether or not the user understands what to do. It is strictly based on whether or not the code functions correctly.

Document Method

Among the 37 responders who said that they previously completed the Google Document method of understanding where they are in terms of pathway completion, the data spread for people's responses can be seen to the left. It's clear that some people view the Google Document method as a very user-friendly experience, but most people view it as a somewhat user-friendly experience.

Website Method

Among the 104 responders who said that they previously completed the Website method of understanding where they are in terms of pathway completion, the data spread for people's responses can be seen to the right. It's clear that some people view the new website method as a somewhat user-friendly experience, but most people view it as a very user-friendly experience.

User Friendliness - Overall

11.9% increase in the extremely user-friendly category.

15.3% increase in the very user-friendly category.

21.7% decrease in the somewhat user-friendly category.

5.4% decrease in the not user-friendly at all category.

2-Prop Z-Test (Statistical Proof)

The purpose of this test is to ensure that my data is not strictly due to random chance. First I ensured that my data met the requirements for a 2 Prop Z Test before actually commencing the statistical test. After making sure that my data met the initial checks, I performed the 2 Prop Z Test. From the test, I found that my "p-value" was 0.0558 with a "z-value" of -1.6. Unfortunately, since the p-value is greater than 0.05 for this test, this does not eliminate random chance as a possible factor in this test, leading to a possible experimental error with my data. This test, however, does not remove my results from above, it just makes them less valid.

Comfortablility

...the well-being of the user while they complete either method of pathway pre-check. Do they know what to do to complete the website efficiently? Did they feel satisfied with the information that they obtained?

Document Method

Among the 37 responders who said that they previously completed the Google Document method of understanding where they are in terms of pathway completion, the data spread for people's responses can be seen to the left. It's clear that some people view the new website method as a very comfortable experience and extremely comfortable experience, but most people view it as a somewhat comfortable experience.

Website Method

Among the 104 responders who said that they previously completed the Website method of understanding where they are in terms of pathway completion, the data spread for people's responses can be seen to the right. It's clear that some people view the new website method as a somewhat comfortable experience and extremely comfortable experience, but most people view it as a very comfortable experience.

Comfortability - Overall

5.9% decrease in the extremely comfortable category.

32.1% increase in the very comfortable category.

23.1% decrease in the somewhat comfortable category.

5.6% decrease in the not comfortable at all category.

2-Prop Z-Test (Statistical Proof)

The purpose of this test is to ensure that my data is not strictly due to random chance. First I ensured that my data met the requirements for a 2 Prop Z Test before actually commencing the statistical test. After making sure that my data met the initial checks, I performed the 2 Prop Z Test. From the test, I found that my "p-value" was 0.00036 with a "z-value" of -3.4. Since the "p-value" is less than 0.05, we have convincing evidence that random chance was not a factor in this proportional comparison.

Information Obtained

...the amount of information, regarding the user's current standing in each pathway offered at GMHS, that they feel they obtained from each method of pathway pre-check.

Document Method

Among the 37 responders who said that they previously completed the Google Document method of understanding where they are in terms of pathway completion, the data spread for people's responses can be seen to the left. It's clear that some people obtained some information, but most people obtained quite a bit of information.

Website Method

Among the 104 responders who said that they previously completed the Website method of understanding where they are in terms of pathway completion, the data spread for people's responses can be seen to the right. It's clear that some people obtained some information, but most people, again, obtained quite a bit of information.

Information Obtained - Overall

7.3% increase in the tons of information category.

8.1% increase in the quite a bit of information category.

11.8% decrease in the some information category.

4.1% decrease in the no information at all category.

2-Prop Z-Test (Statistical Proof)

The purpose of this test is to ensure that my data is not strictly due to random chance. First I ensured that my data met the requirements for a 2 Prop Z Test before actually commencing the statistical test. After making sure that my data met the initial checks, I performed the 2 Prop Z Test. From the test, I found that my "p-value" was 0.198 with a "z-value" of -0.85. Unfortunately, since the p-value is greater than 0.05 for this test, this does not eliminate random chance as a possible factor in this test, leading to a possible experimental error with my data. This test, however, does not remove my results from above, it just makes them less valid.

Additional Feedback

At the end of this survey, respondents were asked if there was any other feedback that they would like to give about the website. Most of the responses were focused on certain honors classes that were missing from the website. It addition to that, however, some users were confused on how the overall process of the website worked. For example, some people didn't know whether they were supposed to enter in all of their classes from their current year of high school or classes from their whole high school career. Others were confused on what classes to enter for strange electives, like AMPED. Many people thought that they were supposed to enter AMPED as well as algebra 1.

Page updated

Google Sites

Report abuse