Computer Aided Assessment - CAA

My interests are exclusively in systems for maths

• which are underpinned by reliable computer algebra packages, preferably ones that I know and use
• which deliver their questions to students via the web

So far, I have authored questions, at various stages of my life for calmaeth, AiM (and, at some future date, I expect I will for stack).

(I debugged a few mapleta questions: could have authored in latex2ta but didn't.)

In 2007, I selected questions for class use from a mathxl question bank.

I really like AiM. AiM's authoring system is the best I have seen in the genre. Nevertheless, I cannot recommend it for any Australian University that doesn't have an unlimited users maple site licence. Although AiM runs with the ALICE java servlet as its ONE maple user, the vendor now wants to count the individual students as users. In any event, the price of the maple part of AiM, from 2008, became prohibitive for UWA Maths (as we are impoverished).

• In 2000, at UWA, I wrote 2nd year eng. maths linear algebra questions in Kevin Judd's calmaeth.
  • In 2001-02 I was employed at University of Birmingham to write questions for the free open-source AiM system.
  • As AiM was also used at Curtin University, it made sense to use it at UWA in areas not already covered by calmaeth as we could use many questions from Curtin's database. AiM's use in the units now called MATH1010, MATH1020 and MATH2040 began in 2004.
  • In 2006 at UWA we explored the use of AiM under moodle for ENVE3605. (This was because moodle was then the Managed Learning Environment used by Environmental Engineering. The 2009 ENVE3605 unit switched into WebCT.)
• Over summer 2006/07,at UWA
  • anticipating price increases for the maple part of AiM from 2008, and
  • anticipating a change of lecturer schedules in 2008 possibly with Kevin Judd and I lecturing MATH2040 together
• I rewrote (several of) my MATH2040 questions into calmaeth, and re-used those of my year 2000 calmaeth questions that were appropriate for the unit.
• Over summer 2007/08, at UWA, I made the (bad) decision to *not* write my own mapleta questions but to try to use the IntroToCalc questions from maplesoft.
• Over summer 2008/09,at UWA,
  • Decision? Scrap mapleta
  • calmaeth for MATH2200 in Sem 1, 2009
• 2011-present: AiM at Curtin University.
• During 2nd semester 2012 I was the main AiM administrator while Greg Gamble, who usually administers it, was on Long Service Leave. Went reasonably well.
• Oct 2012. Paper on hopes for possible use of stack for final year Calculus at High Schools (though the only new user of stack from Conference participants was a Japanese university)
• http://acec2012.acce.edu.au/web-delivered-computer-aided-assessment-maths-stack
• 2015, survey in ACSME15KeadyPaper.
• 2015: demo of stack at Murdoch Uni Maths Dept
• 2018: demos of stack at Murdoch I.T. (private college preparing overseas students for entry to Murdoch)

I think that the way the world is going (whether we like it or not) is that there will be increasing pressure for a neat uniform feel to web-delivered educational material, with Managed Learning Environments (MLEs) - Blackboard/WebCT, moodle as major examples, JellyFish/FlyingFish as Nathan Scott's UWA Engineering local example - playing a role. During 2006, Greg Gamble and I got an AiM/moodle combination going for Math Applications in Environmental Engineering (as moodle is used in Environmental Engineering, and others in the world have developed software so that AiM will run nicely in tandem with moodle). In this vision of the future, the idea is that quizzes would be delivered in the MLE the students at one's university use, and the server with the MLE would commumicate, question by question with servers running (possibly various different) math CAA packages. It may be that this should have been ported to stack/moodle but it was a small class (possibly a good thing for experiments with CAA) and crises with larger lower level classes had to be given priority. The stack/moodle integration is much tighter than is the AiM/moodle integration.

The various CAA packages that I know a bit about

• calmaeth.maths.uwa.edu.au
  • mathematica
  • apache
  • cgi-scripts
  • comments
    • maths via LaTeX turned into gifs, more limiting than some other systems
    • MathML support planned
    • mathematica's pattern matching allows for analysis of student errors, which, I think, is helpful for lower level students.
    • calmaeth was written to run very harmoniously with UWA's Engineering MLE (JellyFish/FlyingFish).
    • in 2015, calmaeth use at UWA was replaced by MathXL/MyMathLab
• aim.maths.uwa.edu.au (discontinued at UWA, end 2007)
  • maple
  • jakarta-tomcat
  • java-servlets
  • comments
    • maths via LaTeX processed by tth
    • MathML support planned, and possible via a combination of maple production of MathML and ttm.
    • the student inputs must be
    • valid maple (and if not, would be rejected, without penalty)
    • valid maple types (specified by the question author, and if not, would be rejected, without penalty)
    • and consequently the marking code has to deal only with "inputs that make sense"
    • I think that the system is particularly appropriate for students who are using, say, Matlab's Symbolic Toolbox (or other maple-derived programs). Cut-and-paste from such systems (with minor edits, e.g. atan or arctan) into AiM basically (mostly) "works".
    • Free, and open source, except for the maple algebra engine.
    • A (fairly major) project in England adapted moodle and AiM a bit, and built a conformity mechanism (RQP = Remote Question Protocol) with the goal that various MLEs would work with various CAA systems. So far it is only for moodle, and basically for stack and AiM. (For details go to mantis york, and choose "ServingMaths".)
    • My eng. maths "linear systems" AiM question code is nice. Should you be at an AiM site and want it, contact me.
• moodle. Not CAA, but MLE Used by UWA Environmental Engineering.
• Math Applications used to be at http://abut.maths.uwa.edu.au
• stack now runs under moodle.
• webwork (and some other systems) communicate nicely with moodle.
• mapleTA: www.maplesoft.com
• trialled, unsuccessfully at UWA, in 2008.
• (A successful system must always mark different forms of correct answers correct and any wrong answer wrong. And the different variants of the questions generated must make sense. The questions selected from a well-rated set of Calculus1 questions failed. They were old EDU ones, lightly modified to sometimes use maple. So the lack of success was due to the question bank selected more than the CAA system itself.)
  • maple
  • I don't know much about server details, but very xml and java oriented, I think
  • comments
    • Its pre-history as EDU has caused it to cart a lot of Managed Learning Environment functionality with it.
    • maple was a retro-fit.
    • There are several authoring systems
      • latex2ta is clearly the best for me to use. Unfortunately, maplesoft support of this has come rather late, as it seemed to promote some other systems earlier.
      • I don't like the GUI authoring from within mapleTA, nor the Maple11on Task "Export as MapleTA'.
      • I much prefer AiM's authoring system to these, but the latex2ta is probably OK.
• WebLearn from RMIT, Melbourne
  • maple
  • I don't know much about server details, but I've heard that many more maple processes run than in AiM so it needs a bigger licence than does AiM
  • comments
    • like calmaeth, this was a local response to building something for maths for a pre-existing MLE (also called WebLearn, I think)
• mathxl from Pearson/Addison-Wesley (successfully trialled at UWA, Sem1 2007)
• not perfect, but being used in 2009 on in MATH1010/1020
  • webMathematica?????? http://company.wolfram.com/news/2002/addison-wesley-introduces-mathxl-and-webmathematica-collaboration/
  • universities run it off Addison-Wesley's servers
  • Comments
    • Was free to universities using a text book with the mathxl questions. Some negotiation needed to achieve this.
    • In North America students get (and pay for) access codes with the text book. Negotiation is needed associated with second hand book sales and new access codes.
    • Huge advantages in that the database of questions lines up with the text, so lecturer selection of questions is facilitated.
    • MyMathLab is related.
• stack:
• From 2012 stack3 lives under moodle.
• http://stack.bham.ac.uk/
• http://stack.bham.ac.uk/stack-dev/lang/en/doc/index.html
  • maxima
  • apache
  • LAMP = linux, apache, MySQL, PHP
  • comments
    • All the comments at the end of AiM apply to stack EXCEPT that maxima - so ALL of stack - is free and open-source.
    • Greg Gamble and I vastly prefer AiM's authoring system (but perhaps we could, if anyone pays us to use stack, augment stack with an authoring system broadly similar to AiM's)
    • I think of stack as a sort of variant of wmi, but adding the assessment functions of AiM, and using tth instead of various alternatives in wmi. (wmi, stack, and aim can all be downloaded via sourceforge.)
    • See also: http://acec2012.acce.edu.au/web-delivered-computer-aided-assessment-maths-stack
    • https://sites.google.com/site/keadystack/home
• CABLE. Developed by Chris Sangwin, who also did stack.
  • AXIOM
  • comments
    • I'm not at all sure that AXIOM rather than aldor was a good idea, for a "sort of CAA like AiM".
    • I would prefer JSP, and Java calling out to aldor functions so that each question was a stand-alone "program" talking to tomcat.
    • As with stack it is all free and open-source
• activemath.org
  • not sure if this project continues as web site has lots of 2011 dates.
  • I believe it is/was underpinned by MuPad which, hasn't been free for over a decade.
  • A further complication is that while MuPad continues as the SymbolicToolbox of matlab, it does not sell in its own right. See https://en.wikipedia.org/wiki/MuPAD
  • WIMS .fr
  • Algebra-Interactive
    • GAP or maple
    • I think via Java servlet in the original version, but I was told JSP in later versions, but I haven't seen it.
    • comments
      • commercial. Goes with Cohen's book of the same name

I know there are others e.g.

• WileyPLUS is underpinned by maplenet.
• (It shares ancestry in EDU, then the eGrade variant of EDU, with mapleTA.)
• I have been through a demo of it, and it seems to work OK.
• the maple-based wallis system from Edinburgh
• Thomas Wolf's one built on REDUCE and used at Queen Mary, London.
• There is a common distant shared ancestry for AiM and for the maple-based system used at UNSW, now discontinued as they changed to mapleTA in 2008.
• I've checked out Brooks/Cole Assessment, BCA, in 2005. This appears to have changed its name since, and, anyway seems to be underpinned by its own Computer Algebra, so I have no idea how good the CA underpinning is.
• WebAssign is another homework hosting service (NCSU). It collaborates with several publishing firms.
• I'm uncertain of how it associates with CA packages.
• WebWork is from MAA, and is free. I don't know which CA package it uses or it it uses its own. I think it doesn't use any CA, but manages to cope with much first year maths because of careful crafting of questions.

I know that there will be many that I don't know of.

Miscellaneous links

• comparison of authoring in calmaeth and in aim: local copy of Jan03 article in MSOR-CAA series on authoring
• nathan's Dec02 article
• http://ltsn.mathstore.ac.uk/articles/maths-caa-series/
• Draft of a talk at UWA Maths, Apr 2004. (This was mostly about AiM, but also a plea to scrap invigilated quizzes in the MCL.)
• UniServeSep06: draft for a talk
• mantis york, and choose "ServingMaths" for more on RQP linking between CAA and MLE