This is a local copy of an article appearing in the 2003 MSOR Newsletter.
Kevin Judd and Grant Keady,
University of Western Australia, AND
Greg Gamble, Curtin University
In this article, we consider the two CAA systems, AIM and calmaeth. Both are underpinned by commercial Computer Algebra (CA) packages. Both have been in use, successfully, for several years.
"It is the best of times": the authors of this article are working at universities in Perth, Western Australia - the 'one city', and even talking to each other occasionally. The arrangement in this article is that Grant Keady, who has authored for both of the 'two CAA systems' treated here, has assembled the article by acting as an "interviewer". The two CAA systems are calmaeth and AIM, its two dialects being aim-plain and aim-tth now available from sourceforge.
Both CalMaeth and AIM are best studied at their web sites, but there have also been various articles in different places. For example, at the LTSN mathstore website we have its CAA series and its Newsletter. Articles in the CAA series include Chris Sangwin's article in May 2002 which contains AIM examples, and Nathan Scott's article in Nov 2002 introducing calmaeth. Neil Strickland has an article on "AIM-tth" in the Feb 2002 MSOR Newsletter.
Kevin Judd is the author of the CalMaeth system, and this system has been used - in its web-delivered form - since 1998. Greg Gamble is employed on a contract at Curtin University to write questions in AIM, further developing the use of AIM at Curtin. AIM has been in use - in Physics as well as Maths - at Curtin University since 2001. (Both these CAA systems have a history extending back to before the web-delivered versions.)
There are many Computer Aided Assessment (CAA) systems delivering, via the Web, questions to students and marking their answers. Some of the systems incorporate Computer Algebra (CA) engines, and this article relates only to two of these. Another one developed in Australia, and in use in Australia, is RMIT's WebLearn system, the maths CAA component of which is underpinned by Maple. Book publishers are also becoming involved with this genre of CAA.
A short table giving some details of AIM and calmaeth is below.
Actually, there are two different dialects of AIM: aim-plain from Belgium, and aim-tth from University of Sheffield. There are inevitable differences in their authoring languages, and we have an example illustrating this later in this article.
Authoring in the systems requires CA competence and HTML competence to author questions. As the questions are likely to be used by hundreds (if not thousands) of students, careful testing of questions is needed.
Here is an example from Chris Sangwin's article earlier in this series:
Example question (17) Give an example of a cubic polynomial p(x) with the following properties
1. p(0)=1,
2. p(x)=0 at x=a and x=b
Here, for each instance of the question given to students, a and b would be small positive integers.
Chris's AIM-plain code: (The p_ in Chris's code is now a_, and the q_ in Chris's code is now b_.)
h> a_ :=rand(1..3)(); h> b_ :=a_+rand(1..2)(); t> Give an example of a cubic polynomial <I>p(x)</I> t> with the following properties t> <OL> <LI> <I>p(0)=1</I> t> <LI> <I>p(x)=0</i> at <i>x=$a_</i> and <i>x=$b_</i>.</ol> v> 4 ap> <i>p(x):=</i> s> [ proc(ans) local marks_; marks_:=1;\ if not `aim/Testzero`(subs(x=0,ans)-1) then\ printf("Your polynomial fails to satisfy <I>p(0)=1</I>.<BR>");\ marks_:=marks_-0.25; fi;\ if not `aim/Testzero`(subs(x=a_,ans)) then\ printf("Your polynomial fails to satisfy <I>p(x)=0</i> at <i>x=%g </i>.<BR>",a_);\ marks_:=marks_-0.25; fi;\ if not `aim/Testzero`(subs(x=b_,ans)) then\ printf("Your polynomial fails to satisfy <I>p(x)=0</i> at <i>x=%g </i>.<BR>",b_);\ marks_:=marks_-0.25; fi;\ if not `aim/Testzero`(degree(ans,x)-3) then\ printf("Your polynomial is not a cubic.<BR>");\ marks_:=marks_-0.25; fi;\ marks_;\ end ,(1-x/a_)*(1-x/b_)*(1-x)] end>
QUESTION TO GREG. Any comments on it?
GREG's ANSWER: Chris's code is in aim-plain, understandable considering when it was written. The Sheffield aim-tth variant is now better established, and, I think it is proper, especially for readers who have already read Chris's article, to see my aim-tth re-working of it.
Greg's AIM-tth code:
d> example of cubic l> a_, b_ h> a_ :=rand(1..3)(); h> b_ := a_ + rand(1..2)(); t> Give an example of a cubic polynomial $p(x)$ t> with the following properties t> \begin{enumerate} \item $p(0) = 1$ \item $p(x) = 0$ at $x = @a_@$ and $x = @b_@$. \end{enumerate} #v> 4 ap> $p(x) :=$ c> polynom(anything, x) s> [ proc(ans) local marks_; marks_ := 1; if not `aim/TestEqual`(subs(x=0, ans), 1) then `aim/t`("Your polynomial fails to satisfy $p(0) = 1$.\\\\"); marks_ := marks_ - 0.25; fi; if not `aim/TestEqual`(subs(x=a_, ans), 0) then `aim/t`("Your polynomial fails to satisfy $p(x) = 0$ at $x = @a_@$.\\\\"); marks_ := marks_ - 0.25; fi; if not `aim/TestEqual`(subs(x=b_, ans), 0) then `aim/t`("Your polynomial fails to satisfy $p(x) = 0$ at $x = @b_@$.\\\\"); marks_ := marks_ - 0.25; fi; if not `aim/TestEqual`(degree(ans, x), 3) then `aim/t`("Your polynomial is not a cubic.\\\\"); marks_ := marks_ - 0.25; fi; return marks_; end,(1-x/a_)*(1-x/b_)*(1-x)] sb> t> Observe that a polynomial $p(x)$ that is the product of linear factors t> $x - 1/x_i, i = 1, \ldots$ satisfies $p(0) = 1$ and has zeros at t> $x = x_i$. Thus t> \[ \left(1 - \frac{x}{@a_@}\right)\left(1 - \frac{x}{@b_@}\right)(1 - x) \] t> satisfies $p(0) = 1$ and is $0$ at $x = @a_@, @b_@$ (and $1$), and since t> it is the product of 3 linear factors it is a cubic, and hence satisfies t> the required criteria. se> end>
QUESTION TO KEVIN. How does the equivalent code go in calmaeth?
KEVIN's ANSWER: Calmaeth allows unlimited variety of question types and very sophisticated diagnostics, but at present to use these requires special purpose-built GUI tools. To make simple questions I introduced a collection of flexible generic question types, which allow multiple parts and multiple variants of each part. CalMaeth is built around the Jellyfish database standard. The GUI tools mentioned above just manipulate the database files, but simple questions can be edited by hand and on-line with a web browser. The following database implements the questions. I wrote this by hand. CalMaeth also allows generating stuff with latex, but for clarity I do not do this here. CalMaeth can also get Mathematica to generate GIFs of graphics too.
# First some stuff to define question type, seed, marks, assessed type=Generic/1.2/InputTest seed=1037005936 marks={1} assessed= # Declare variables we might allow in expressions variables={x,z,t} # Initialize some $variables# Note: once something is initialized, use $variable to use it init/x= GenSelection[{x,z,t}] init/a= Random[Integer,{1,3}] init/b= $a + GenSelection[{1,2}] init/p= (1-$x/$a)(1-$x/$b)(1-$x) # Questions can have variants we can select,# here there is just one, so select it init/select={"1"} # State the question# Note: one-line definitions use =, multi-line definitions use <<...>> question<< <em>Give an example of a cubic polynomial p($x) with the following properties<br /> 1. p(0)=1,<br /> 2. p($x)=0 at $x=$a and $x=$b </em> >> # Details of variant 1 (the only variant in this case)# Note: variant things prefixed by variant name ("1" in this case)# Define and label input box 1/statement=Enter your answer here. 1/type=Expression 1/detail=<i>Answer is a Expression</i> # This is the correct answer for "Solution" page. 1/correct=$p # Now tests to apply to student answer# In each case: # apply test, if True then # text is reported # mark is reduced by severity percentage (no negative # marks)# If RHS of test is just an expression,# then just test student's answer is the same, report text if same# Note: copes with expanded and unexpanded polynomials 1/Correct/test=$p 1/Correct/text=Correct! 1/Correct/severity=0 # If RHS of test is a pure function,# then apply function to student's answer 1/test0/test= (!PolynomialQ[#])& 1/test0/text= Your expression is not a polynomial! 1/test0/severity=40 # If RHS of test is a held expresion,# then substitute "answer" for student's answer, evaluate and act as above 1/test1/test= Hold[(answer/.{$x->0})=!=1] 1/test1/text= Your polynomial fails to satisfy p(0)=1. 1/test1/severity=20 1/test2/test= Hold[(answer/.{$x->$a})=!=0] 1/test2/text= Your polynomial fails to satisfy p($a)=0. 1/test2/severity=20 1/test3/test= Hold[(answer/.{$x->$b})=!=0] 1/test3/text= Your polynomial fails to satisfy p($b)=0. 1/test3/severity=20 1/test4/test= Hold[Exponent[answer,$x]=!=3] 1/test4/text= Your polynomial is not a cubic. 1/test4/severity=20 # Just being paranoid, or no one is perfect# Severity of -1, means student does not get penalized! 1/CatchAll/test=True 1/CatchAll/text<< Something really weird has happened. Please report this error! >> 1/CatchAll/severity=-1
There are translation tools, Maple to Mathematica (and vice-versa).
QUESTION TO KEVIN: Where AIM's question types correspond to those in a subset of calmaeth, do you consider translation tools - for h> and t> fields - (i) possible, (ii) potentially useful?
KEVIN's ANSWER: (i) I think it is possible. CalMaeth's questions, like the generic question type of the example above, use a database approach, which only requires snippets of HTML and Mathematica code in the right places. I think I could do more than you ask; I could write a new question type that mimics the AIM way of doing things - the whole question file, all flags: this does not look difficult to me. This reduces the problem to just converting Maple to Mathematica.
(ii) If some CalMaeth author were to think that an AIM question type would be potentially useful to them, I would consider doing it.
One concern is that Maple and Mathematica are possibly both too large (or may become too large) and too general purpose for the low-level algebra demands of them appropriate in the context of 1st year undergrad. maths as AIM and CalMaeth are currently used. In particular, if software of this genre were to be used in British A-level, or Australian "Tertiary Entrance Exams", it is arguable that it is overkill to be using Maple or Mathematica. Bronstein's "BasicMath" library in aldor (www.aldor.org) is possibly more appropriate for authoring in (at least some parts of) this genre than are the large general-purpose Maple/Mathematica systems. (See also http://home.earthlink.net/~jgg964/axiom.html and/or http://savannah.nongnu.org/projects/axiom/ for AXIOM-related items.) Of course, a virtue in using the mass-market Maple/Mathematica packages is that lots of people can author in AIM/CalMaeth without learning other CA software.
QUESTION. Could one replace the existing commercial CA underpinning your CAA with another CA package, e.g. AXIOM/aldor?
KEVIN's ANSWER: In principle, no. The CalMaeth server just manipulates databases and template documents. When it generates a random question, or checks an answer, it parses a template Mathematica expression, gathers up information from various databases, plugs the information into a template expression, which it sends to Mathematica, and finally unpacks the results into the database. The feedback HTML the students gets is formed by substituting information from the database into a template feedback HTML. So in principle any auxillary engine could be connected to add new features: indeed, I once investigated connecting GAP, but there was no real demand for it. On the other hand, a lot of existing CalMaeth questions exploit Mathematica's pattern matching capabilities for their sophisticated diagnostics, and this would be hard to emulate.
GREG's ANSWER: Not really. AIM has been written with Java doing relatively little, and Maple doing a lot. Maple writes the HTML pages.
In principle, a redesign using Java Server Pages, and confining the use of the CA system just to the "maths" would allow for alternative CA systems to be used. However, even if this were done, one would be asking quite a lot of the translation tools, Maple to whatever. These will attend to trivial matters, like the optional * for multiply in Mathematica. The translation tools will also attend to some of the function-name differences, number of arguments or ordering of arguments of functions, and syntactical differences. However, to give a reasonable example, one frequently uses Maple graphics in aim's p> fields, and the differences between Maple and Mathematica graphics commands are not well handled with the present translation tools. And, as regards other CA, like AXIOM, I'm not aware of any translation tools, Maple to AXIOM.
KEVIN's ANSWER: Mathematica produces MathML, but browsers that support MathML have only just been released. My tests show that the integration is not complete. I think we will have to wait for the next generation of browsers, perhaps in a year or two. CalMaeth is over due for an upgrade (now scheduled to begin 2004 because of other commitments), the next version of CalMaeth will be MathML compatible.
GREG's ANSWER: Maple produces MathML. AIM-tth currently has its user's entering LaTeX fragments and Maple producing LaTeX, and the rendering done by tth. Neil Strickland has designed AIM-tth keeping in mind a replacement of tth by ttm (ttm = TeX to MathML) - the resultant AIM-ttm would have Maple producing MathML (directly) and LaTeX fragments by quiz authors translated by ttm to MathML. This change is likely to occur once most web browsers are MathML-capable.
Of course, this would offer some scope to use well-tested code for the logins, collection of marks, lecturer administration tools and so on. Of course, the present CAA systems - AIM and CalMaeth - have tools which, at least in some areas, are better than those in WebCT.
KEVIN's ANSWER: I have no immediate plans to link calmaeth with WebCT.
GREG's ANSWER: I know of no immediate plans to link AIM with WebCT.
In one subject at Curtin the students were given email access via WebCT and so WebCT provided the email/discussion fora and was where marks were eventually, though the quizzes were at the olld site. We are considering modifying the login for AIM so that it authenticates students in the same way as WebCT (which we believe should be fairly easy to do).
KEVIN's ANSWER: No immediate plans for its use. I'm aware of webMathematica use in mathxl.
GREG's ANSWER: No immediate plans for its use. I'm aware of the references to Brownstone Learning's CAA in connection with maplenet. I am in the process of checking out maplenet via the demonstration site at maplesoft. There are, of course, other uses of maplenet - and web-delivery of maplets - than just CAA.
GREG: CAA = Computer Aided *Assessment*. As discussed elsewhere in this series and on the LTSN site, there would be a real danger in cost-cutting administrators urging us to overuse CAA as a REPLACEMENT for traditional teaching methods. Its role is as a SUPPLEMENT, and, to be educationally effective, CAA must be used intelligently. AIM-tth quizzes can be set up to operate in one of 3 modes: practice, assessment and strict; `practice' mode allows a student to do a quiz over and over, allowing the students to see solutions for the questions - fresh accesses to the quiz after seeing the solutions generate new versions of the quiz. `assessment' mode is by far the most useful - students have the incentive of getting a mark for their attempt of the quiz, and are able to redo questions they get wrong; if the quiz author has been sufficiently diligent the students derive a great deal of benefit from feedback produced by wrong answers and so the students learn from their mistakes. `strict' mode allows only one attempt of each question of a quiz.In `assessment' mode, one can expect many (hopefully most) students to get full marks for the quizzes; in this way, it is not a discriminating assessment tool; it is really a way of ensuring students do regular work and keep up with lectures. This is an appropriate usage of the technology which should be used to complement traditional tutorials and assignments, since CAA quizzes only look at the `bottom' line (the answer line) of a student's solution and not the reasoning that led to it. In today's world the body of a solution can only reasonably be assessed via tutorials and written assignments. 'Generic' skills, like being able to write and speak good English when explaining one's mathematical solution are very important skills, and best assessed in ordinary written assignments, and with student presentations in tutorials.
Some of Curtin University's materials are available at Curtin's AIM server
http://olld.maths.curtin.edu.au/
Guest logins logins are possible by selecting `guest', typing `guest' in the `Your student ID' box, leaving the password blank and clicking on `Login'.
GRANT: Thank you, Kevin and Greg. CAA systems will continue to evolve. AIM and calmaeth both demonstrate that they are already reliable. I predict mass-usage of mathematical CAA, e.g. in upper levels of high-school, before the end of the decade. I'm sure we all agree with Greg on the importance of the students having skills in writing mathematics on paper too.
Nevertheless, as is appropriate at the end of a "Tale of $n$ cities": "It is a far, far better thing", for sufficiently large, sufficiently low-level classes, to have the computer mark some of the many thousands of routine calculations, than to have us over-worked lecturers burdenned with all of the drudgery.