Last Wednesday, I passed my final defense. I am now Dr. Bart. Onwards and upwards!
Something bothered me when designing our last syllabus. The final product looked something like this:
This table breaks down the apportioning of points by course component. But this doesn't reflect the goal of my teaching. I am not concerned how my students do "on classwork". I want to know if they understood the actual topics that I derived from my instructional analysis. Ideally, I would be apportioning percentages for content areas:
At some point over the semester, my students must convey to me that they sufficiently mastered the concept of how programs can loop over a list, and then they should receive some points. I want them focused on completing the topics, not "classwork".
This thought was triggered again tonight when I saw a presentation complaining about how Mean grades can be misleading. They offered an example similar to the following:
They argue that, clearly, assignment 4 was an outlier that should have been removed from the final grade calculation. This student consistently performed above a 90% with only a single exception, and clearly something was wrong then. But if you're using the first syllabus I offered, then perhaps Assignment 4 was the activity on loops. It's not surprising that they would do poorly on this subject, even if they did well on much of the rest of the course content. And yet that topic is weighed as just a simple piece of the final Classwork grade. If it were the lesson on loops, then they should find it very concerning that they don't understand the material, since its so important! I would like the grading scheme to reflect that importance.
Obviously, this should be balanced with mastery-based grading; give them as many attempts as they need, within a reasonable amount of time. Give them the lesson in as many different forms as they need. These thoughts aren't about punishing students for not performing perfectly. It's about getting them to put energy where I feel it is most valuable.
Harry Potter and the Sorcerer's Stone has, roughly, 5746 sentences, spread out over 17 chapters. I know this because I have recently tried making a new style of English dataset for the CORGIS project. Previously, we had attempted to create datasets for English majors by analyzing a corpus of books and computing some statistics (difficulty, sentiment analysis, etc.). This new approach would look at a single book, which actually gives us the ability to compute some new statistics.
One of these statistics is particularly interesting: Tone Analysis using BlueMix from IBM Watson. Essentially, this service allows you to get a variety of tonal information for a sequence of sentences. For instance, you can get the estimated "Sad"ness of a sentence as a decimal 0..1 value, along with a number for "Joy", "Fear", "Anger", and "Disgust".
So what happens when you put in the text of the first Harry Potter book?
Unfortunately, that's not particularly helpful. The results of tone analysis fluctuates so much over just a few sentences that we end up with a mess of lines. I believe that the only reason its purple is because that was the last tone drawn.
It is fairly easy to create a Rolling Average with Pandas+Scipy+Numpy. This helps tease out a more useful graph, especially when the tones are standardized.
Now, it's not a perfect match up, and there are a lot of other trends to explain here. But I did think it was interesting to see how we could pair up important events with peaks and troughs. Unfortunately, all the postprocessing I had to do makes this kind of analysis out of the reach of most of our students without some more help. I have to carefully think about how to make this kind of data available in such a way that its still useful. Dataset creation is tricky business!
In the Computational Thinking course, we talk about "Abstraction" as the concrete representation of real-world things using computational tools (although there are other definitions, this is what we focus on). Variables are one way to represent things, but there are others. Lately, I have been considering how different variables have different levels of abstraction, and the implications for pedagogy.
Let's walk through an example. Imagine we have a herd of cows in a field.
This herd could be represented quantitatively using the following table.
At the bottom row of this table, we see an "average" cow. There is no "average cow" in our herd with a weight of 716 pounds. But we understand this idea of an "average cow" as an abstraction.
Now, how would we represent this herd in a BlockPy program?
We now have a variable that abstracts our herd of cows into a list of numbers, which can be easily manipulated. By abstracting our cows, we reduce unnecessary information (their height, their name, their genetic code, how many times they've mooed today, etc.).
We can create a variable that represents our imaginary average cow.
We cannot see the value of the "average_cow" from this perspective, although we could print out its value. It is a more abstract variable, dependent on time and other pre-existing variables.
We used built-in Python functions to quickly compute that average, but in the CT course, we don't allow students to do so. Instead, they have to use a for-each loop. The necessary code to calculate the average cow weight would be:
This code has 5 variables:
The variables "total_weight" and "total_cows" are similar to the "average_cow" variable, but they are at a higher level of abstraction than "my_cows" since they do not represent real physical entities.
And what about that "a_cow" variable? It represents each cow, but only over the entire course of the loop. It is not a specific cow until a specific time. To me, this represents an even higher level of abstraction than the other variables.
Looking at a completed program, I think many students are not able to recognize the varying levels of abstraction of these variables, and they struggle with writing this all from scratch.
In the past year, I've had a growing interest in formal methods of Instructional Design. One of my new favorite activities is writing learning objectives. I'm still developing that skill, but I though it would be interesting to share some of the objectives I've written for the Computational Thinking course. As you can see, there are a large number of learning objectives, and I doubt we actually cover all of them in the CT course. There's still a lot to improve about the curriculum. There's also a lot to improve about these outcomes. I noticed while reviewing them that there isn't anything about students working with nested control structures, for instance.
I've decided to try and share some introductory materials for getting into teaching. Mark Guzdial announced that he'd be posting the syllabus for his CS Teaching Methods course, and recently two CS people have asked me how to get into CS Ed research. Clearly this is the hot time to be dragging people into the best CS subfield.
The fact is that there isn't too much published Pedagogical Content Knowledge about how to teach Computer Science. There's a lot of Pedagogical Knowledge out there, and as CS people we already have Content Knowledge. But the community is still establishing what's different about teaching CS compared to, say, teaching someone how to make Cigar Box Guitars.
So most of this page is devoted to good teaching links, and some specialized CS information. At some point I may toss out my own model curricula for a "CS Teaching Methods" course. There'd probably be a lot of theory (Instructional Design, Gange's Learning Events, Motivation, SL Theory, Constructivism, Cognitivism, etc.) and some large amounts of practice.
Formerly http://www.learningandteaching.info/ (still available through https://web.archive.org/web/20160212181036/http://www.learningandteaching.info/)
Hands down the best resource I've ever found for growing as a teacher/researcher/learner is this website. Unfortunately, it went down a year or two ago, for some reason. But you can still access it through the wayback machine!
The site is divided into three sections:
I recommend starting in column 1 if you are new to education research, and starting in column 2 if you are eager to start teaching and building curriculum.
There are many theories related to learning, teaching, motivation, etc. This site gives you a crash course in all of them. It is overwhelming at times, and learning how to navigate these theories is tricky. Over time, you should come to view learning theories as lenses. Different theories are valuable at different times. For example,
There's a lot more out there than just Behavioralism, Constructivism, and SL Theory; although if you stick to reading CS Ed literature, you might think those are all there are. Don't get caught in that trap, but explore as many theories as possible and see how they can help you. It's like Object-Oriented vs. Function programming: different tools at different times.
There is a tremendous amount of research out there about how to teach CS. As a community, we're till trying to distill the best of it out, but you should definitely read some of these proceedings and journals. In particular, look at:
I do a lot of motivation research, and the MUSIC model is by far one of my favorite theories. I'm biased, since Dr. Jones is on my committee, but I asked him to be on my committee because it's a good theory, so there you go.
I don't agree with all the tips that are here. Some of them are quite interesting, some of them are quite useful. But definitely take them with a grain of salt.
Google has a really nice sample curriculum here for teaching Computational Thinking to K-12. I think it's a nice model of what we should be building as a community.
There is no teacher like experience. You can read guides all day, but honestly you're better off getting real world experience as quickly as possible. Teaching is one of those domains where Situated Learning techniques really pay off. So work with a local high school to teach a class or something.
As I write my prelim, I'm re-immersing myself in Situated Learning Theory. I've been obsessed with Instructional Design for the past few months, so I haven't thought much about this alternative educational theory. However, I started using it early in my graduate career for a reason, and those reasons still make sense. I'm using a particular evolution of SL Theory  that breaks down the learning process into four components: the context, the content, scaffolds, and the assessment. In the latest iteration of my prelim, I'm crossing this with the MUSIC model of academic motivation to explore how the components provide opportunities to motivate the learners - in particular, what the context and scaffolds bring in.
While working this out, I made the following table to organize my thinking. I thought it was an interesting break-down of where motivation could apply within a learning experience. There's a lot more not captured here in a learning experience, but this is what was useful for my research.
This is only a small part of what I'm doing in my prelim overall. For instance, there's all the fancy software I'm writing to fill out the scaffolds column. Still, I think this is an interesting way of looking at the components of the course.
 Choi, Jeong-Im, and Michael Hannafin. "Situated cognition and learning environments: Roles, structures, and implications for design." Educational Technology Research and Development 43.2 (1995): 53-69.
I'm extremely lucky to be funded by the NSF Graduate Research Fellowship Program. One of my obligations to the NSF in light of this is a write-up of my activities for the past year. I thought it would be interesting to post my write-up here. I apologize for its rough quality - I wrote it in one-sitting!
This past year, I have been primarily engaged in creating a new undergraduate course on Computational Thinking meant for non-Computer Science majors, assisting Dr. Dennis Kafura. This course uses an innovative Data Science approach, requiring us to create an entire curriculum from the ground-up. Students get a chance to grapple with data from their own discipline in a collaborative learning environment with an emphasis on mastery learning and authentic learning experiences. In Fall 2014, we ran a pilot course for 20 students from over 10 different majors - in Spring 2015 we have 40 students from over 20 different majors, including a 50%/50% split in gender. The main challenge of this course is scaling up the learning experience to eventually handle hundreds of undergraduates per semester, many of whom will struggle with the material and lack proper motivation - we accomplish this by using innovative pedagogy such as collaborative student cohorts and active learning techniques. As we develop this course, we have an eye towards wide dissemination of the course components even as we try and scale up the material. In addition to a publication at ITiCSE on the pilot offering, we were written up in a local newspaper for the novel pedagogy and technology.
In order to help run this course, I have created a suite of new educational technology resources. The most valuable item is a new block-based programming environment that scaffolds students to write real Python programs - these programs can be converted back and forth freely between the block-based representation and the conventional text-based version. This environment also gives students interactive feedback as they complete assignments, guiding the student through the material and reducing the workload on the instructors. This environment is embedded within a new online learning platform that gives students an immersive experience as they complete coursework - all of their grades and assignments are in one convenient web interface, and they can rapidly switch between classwork and textbook readings. From the instructors perspective, this system tracks individual students' progress across a number of factors in order to keep track of a large number of learners. Finally, I have continued my work for the CORGIS project: the Collection of Real-time, Giant, Interesting dataSets. This project seeks to make high-quality Big Data sets available to novice learners through a simple-to-use interfaces. At present, we have over 3 dozen ready-to-use libraries across a wide-variety of subjects, empowering students in the Computational Thinking class to find a dataset that speaks to their interests and long-term career goals. Students use these libraries through the block-based programming environment and during their final project, where they must answer real-world problems using computational techniques and a datasets of their own choice. All of these resources are free, open-source, and publicly available.
In addition to my work on this new course and its associated technologies, I have been involved in several smaller research projects and professional development activities. Working alongside my fiance in the Animal Science department, we have created an educational modification of Minecraft meant to organically teach Animal Science concepts ("AnimalScienceCraft") aligned with FFA standards for middle schoolers - although still in the early stages and seeking funding, we have already presented this research and garnered a lot of positive interest. I have committed many hours to supporting and hosting events for the Computer Science department's Association for Women in Computing, in my capacity as an officer of the organization - this work is critical to my on-going commitment to promoting women in computing. As a senior graduate student, I have mentored two of my fellow graduate students and two undergraduate students as they conduct research within my ken. At one of the largest conferences in my field (SIGCSE'15, the Special Interest Group on Computer Science Education), I organized a PhD students dinner in order to foster community among the CS Education doctoral community. Finally, I am also serving as Publicity Chair for the education workshop for SPLASH (SPLASH-E'15). I hope to continue many of these efforts even as I continue to develop the Computational Thinking course, conduct informative empirical studies on its participants, and create powerful new educational technology to support it.
On Wednesday 2/4/15, I attended Chep'15 here at VT.
Although there was a lot of great stuff there, one panel stood out:
In Computer Science, we have a big problem with self-regulated learning. I've seen so many students confident that they can do the entire project at the last minute, and then have to email the instructor 10 minutes before it's due with basic questions. In theory, students should come to college with these skills - in practice, this is where they learn them.
The slides of the talk are here. I've made my own notes on the most interesting things below. All credit for these ideas goes to Dr. McCord.
Successful usage of SRL strategies is correlated strongly with success in a course. Can we really modify student's behaviors in order to get the benefits of good SRL? I want to see more research on what implementing these strategies does for students in terms of learning gains.
By Austin Cory Bart and Eleonor Cayford
Special thanks to Dr. Eli Tilevich and Dr. Michael Evans
This past year, Ellie and I have been collaborating on an educational Minecraft mod to teach Animal Science. This blog post represents a summarization of the work that we've done, along with the future work we have planned. It should be appropriate for both researchers and non-researchers who are interested in this project.
Minecraft is an exceptionally popular video game. Earlier this year, it became the best selling PC game of all time, beating out the Sims (despite being much younger) . The success of Minecraft lies partially in its novel core game mechanics; a fully interactive, infinitely generating, block based world . However, it has also developed perhaps the most sophisticated modding community in the history of gaming. Not only are the core game developers constantly engaging with players, but they are very permissive about people extending the game with new mechanics, blocks, and everything else under the sun .
One of the most interesting applications of Minecraft is to education, and there have been many interesting projects  . For instance, several Computer Engineering classes have used the RedStone mechanic of vanilla Minecraft as demonstrations or even for assignments in Digital Logic (building simple machines and even fully fledged CPUs) . Mods bring even more opportunities for education. The successful Minecraft.edu project support teachers with classroom management tools, lesson plans, and other useful utilities all baked into Minecraft . Here at Virginia Tech, the ICAT OperaCraft project enabled high schoolers to author, choreograph, and perform a digital opera . And now Foster and Esper (the same amazing duo that made CodeSpells) is working on a Blockly-based modding system for Minecraft (LearnToMod) to teach kids programming. There are many other projects, and the research applications are blossoming.
It is exceptionally easy for people to misestimate the value of video games for education. On the one hand are the hopeful, who see software as the magic wand to fix education - automated assessment, just-in-time feedback, and rich motivating experiences. On the other hand are the bitter, who have seen games like Math Blaster and how little impact they really have on children's learning. Educational game development is difficult, requiring pedagogy, art, music, game mechanics and everything else to come together in a perfect mixture. Research is required to really know that any given educational game is successful. Still, the literature offers many starting points and lens to analyze games. For the Platipy project, I have previously written summaries of several research papers on educational game design, and I'd like to suggest anyone interested in vanilla Minecraft for education read the Charskey et al paper.
More relevant to this discussion is a second paper by Squire, describing two different paradigms in educational game design: exogenous and endogenous. To quote: "In Exogenous games, the learning context is external to the gameplay, as opposed to internalized like in Endogenous games." In an endogenous game, understanding the material is key to succeeding in the situated game world at a very deep, integrated level. In an exogenous game, the educational material has been grafted on - consider a Mario game where every few minutes you are interrupted with a multiple choice quiz that you must pass before continuing the real game. To be successful, game designers should focus on developing endogenous games (unless drill-and-practice truly is all you need). When designing AnimalScienceCraft, Ellie and I constantly tried to keep things Endogenous.
When I first got Ellie to play Minecraft, she was largely enjoying herself up until we started working with animals. The existing animal system is a giant facepalm for an animal scientist, with many things that are just plain silly. Why is genetic inheritance so lame? What is in the wheat that induces fertility? Obviously, these don't make the game unplayable, and they are perfectly adequate game mechanics. However, there is a lot of untapped potential here to convey some interesting scientific topics through a really exciting 3D world. Raising animals in Minecraft is a useful activity - they give you food and resources that are otherwise rare. Ideally, optimizing the process should require students to understand how the true process works.
Animal Science is a very important field, but they're a little glossed over in most discussions of STEM. Like other science and engineering fields, it can be difficult to draw in students. We're hoping that an engaging game will improve visibility of the field and the value it provides to society. We're also hoping that the embodied nature of the learning experience will be more valuable than a comparative experience of reading a textbook. To that end, we decided to work on AnimalScienceCraft, a modification for Minecraft that introduces new mechanics intended to teach core concepts from Animal Science and that motivate students to learn more about Animal Science.
This work began under Dr. Michael Evans when I took his Digitally Mediated Learning class. Dr. Evans organized his course around a semester project, and this led to the design document and initial research for AnimalScienceCraft. The following semester, I took an independent study with Dr. Eli Tilevich in order to develop an actual prototype.
Before beginning work on educational games, it is critical to know what your learning objectives are. We used standards developed by the National Council for Agricultural Education National Council for Agricultural Education, designed for middle- and high- school aged students. This list is actually fairly massive, and covers a lot of agricultural material outside of Animal Science. We only concerned ourselves with the subset of Animal Science objectives that we could transform into meaningful gameplay elements, to avoid cognitive overload. The relevant list of objectives are summarized in the table below:
In the following section, we'll describe the mechanics we've introduced. There are a lot of changes to the core game here!
Objectives: Reproduction, Classification
Animals now have Gender, which results in visible differences in the animal (in the picture above, male pigs are darker than female pigs). In order to breed two pigs, they must be different genders. Further, gender affects the utility of the animal - male cows do not produce milk, for instance.
Female animals have an estrous cycle now - once they reach adulthood, they will periodically go into heat (indicated by hearts over their head). At this time, they will actively seek out a male to mate with. Two animals are needed to reproduce:
Objectives: Classification, Inheritance, Trait Breeding
When animals reproduce, their new genetic traits are based on their parents. There are four genetic traits:
Unlike regular Minecraft animals, they tend to improve over multiple generations - however, animals need to be bred to specialization, e.g., breeding for a higher Meat Quantity leads to a lower Potential for Production. Every new generation of animals, you will expect some distribution of animals - you need to cull the lesser animals, and encourage the good ones. When you right-click an animal, a window pops up with information about them - eventually, you can choose to have a specific trait visible over all the herds (so you can eyeball your herd's current Meat Quantity for example).
Objectives: Animal Handling
Animals now eat to survive. If their hunger drops to zero, then they die. The amount of hunger they recover from their food source is determined by their Feed Efficiency stat.
Animals products dropped on death have better return on investment. This isn't meant to teach - it's an incentive to raise animals. Death drops are determined by the Meat Quantity stat, with a higher stat yielding more and possibly better meat:
Some animals have by-products that do not require killing the animal. These drops are determined by the Production Potential stat.
Size and Age
Animals age and die - but on the plus side, they are now invincible to enemy mobs. As animals get older, they increase in size. This size is also determined by their hunger - hungrier animals are leaner and smaller. Finally, baby animals no longer have comically oversized head. We argue they look cuter this way.
Objectives: Reproduction, Inheritance
Breeding related animals in the real world (inbreeding) can lead to genetic problems because recessive problems tend to build up and lead to inferior offspring. Lineage is tracked for each animal, and they will not breed with another animal that has a common ancestor (to two generations).
Animals are named based on their species and reproductive capabilities. The in-game system uses this taxonomy. This knowledge is situated in the lingo of the game, so ideally users should start picking it up naturally.
Wild animals are not easy to manage - they are skittish if you move too fast and they might even attack if you approach from the wrong side. Domesticate animals by breeding together more passive animals - most packs will have at least one chill animal in the group, and they are the ones you should pick. Once animals have been bred for domestication, they can be bred for Meat Quantity, Production Potential, and Feed Efficiency (this is why you will not see wild pigs with exceptionally high Production Potential).
Objectives: Reproduction, Handling
Stressed animals will not reproduce and will get antsy (make them move about more). They also act like they are less domesticated. One of the most complicated aspects of animals are their stress level, which is a combination of many factors. For instance: animals do not like to be too tightly packed, they do not like to be hungry, etc.
I am midway through a PhD in Computer Science and Ellie has taken an introductory course on Computer Science, so we have a large amount of programming expertise. However, Minecraft modding is pretty difficult - tutorials and reference materials are hard to find and are frequently out of date. The community has been extremely helpful in answering questions, but it can still be a little overwhelming at times. I'm sure that a number of our design decisions were bad, so feel free to criticize our architecture. Long term, I expect to throw this alpha version out and start fresh for version 1.
Our mod was created using Forge for Minecraft 1.7. A powerful tool, Forge provides a useful skeleton and a nice event system. Of course, there are limitations: most identifiers (the names of properties, methods) have been crowd-sourced, but coverage isn't 100%. There isn't exactly exhaustive documentation, either. At one point, I was trying to make a pig jump on demand (I can't recall why). I found methods with "jump" in their name, and a JumpHelper class, but after hours I couldn't seem to affect the pig - eventually, I discovered you had to change the y-velocity of the entity. Certainly makes sense in retrospect, and I wish I had simply asked someone on the Forge forums, but our work was replete with such hassles. I often felt like I was debugging before I even wrote any code!
In order to introduce our large changes, we extended EntityAnimal with a new abstract class named EntityScientific, which in turn was extended by each of our animals (currently, only EntityScientificPig is ready). We then deregister the regular animal spawns in favor of our own. This was a difficult decision to make, because this breaks compatibility with other mods that rely on the specific vanilla animals - EntityPig, EntityCow, etc. On the plus side, existing mechanics largely refer to EntityAnimal, so we still can use things like tethering, eggs, etc.
Minecraft has a client-server architecture, even in local singleplayer: the server is largely responsible for managing world state (and is the only place where persistence occurs), while the client is largely just handling rendering. However, except in specific circumstances, all code is run on both the client and server (reminiscent of what happens when you fork a process), requiring some checks to see who's running the code. Communication between the client and server is handled by either the networking system (which I was hesitant to dive into on my first attempt) and the DataWatcher (a reflected set of variables that are automatically synchronized between client and server). We didn't realize how crucial this communication was until we had already created a large number of instance variables to keep track of our animals' many properties, when we discovered that nothing was persisting between runs. The simplest solution was to store this information into the DataWatcher and let it handle everything. The DataWatcher is a strange beast, allowing only 32 variables and only of certain types: int, float, byte, char, a few special Minecraft types, and String (maximum 32767 characters). This last was a surreal discovery to me - you can fit 16k integers into one of those Strings! It was a quick slippery slope to create my own stringly-typed tuple to get past the 32 variable limit. I'm largely certain I'm wasting a lot of bandwidth with this approach, so it probably needs to be revisited.
All work for this project is freely available on the LearnCraft github. I would describe this mod as being roughly "pre-alpha". At the time of writing, only pigs and cows are included (and not all features described above are developed). However, many of the core features have been developed. Still, a lot more work is required before the system is ready for an intervention with real students.
Will AnimalScienceCraft be successful? We won't pretend that ASC is something that everyone will add to their installation of Minecraft and then play in an informal setting - it introduces some changes that might make the game a little less fun at the cost of the educational value. However, I can easily imagine it being used in a formal (e.g., middle school science class) or non-formal setting (after school FFA club). People with a casual interest might play it a few times and pick up some knowledge that they didn't have before.
Our next big step is to get funding for this project, possibly through the ICAT center at Virginia Tech. Not only will we be able to scale up development, we can also conduct formalized research in order to find the real value of this system. We really need a dedicated undergraduate or graduate student who can develop on it full-time. Digitally Mediated Learning offers a lot of promise to teach people in innovative ways, but it rests on researchers to ensure that the tools really stick the landing."Minecraft Stats". www.minecraft.net. Microsoft. Retrieved 20 September 2014
 Wikipedia contributors. "Minecraft." Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 19 Dec. 2014. Web. 26 Dec. 2014.
 Mojang. "MINECRAFT END USER LICENCE AGREEMENT." Mojang. Web. 26 Dec. 2014.
 Phu, L. "Minecraft in the Classroom: When Learning Looks like Gaming." KTOO. 24 Apr. 2014. Web. 26 Dec. 2014. <http://www.ktoo.org/2014/04/24/minecraft-classroom-learning-looks-like-gaming/>.
 ThatWeirdPhysicist. "Learning logic gates in Electronics Class." Imgur. May 5, 2014. Web. 26 Dec 2014. <http://i.imgur.com/CCQiVmj.jpg>. "MinecraftEdu - About." MinecraftEdu - About. Web. 26 Dec. 2014. <http://minecraftedu.com/about>. "OPERAcraft." OPERAcraft. Web. 26 Dec. 2014. <http://www.icat.vt.edu/funding/operacraft>.
 OPERAcraft. (n.d.). Retrieved January 12, 2015, <http://www.icat.vt.edu/funding/operacraft>
 Short, Daniel. "Teaching Scientific Concepts Using a Virtual World--Minecraft."Teaching Science 58.3 (2012): 55-58.
 National Council for Agricultural Education. (2009). National agriculture, food and natural resources (afnr) career cluster content standards. Alexandria, VA: National FFA Foundation.