AR Platforms

Outside of our use of existing platforms (outlined below), our project has planned what an ideal AR app would look like. Our approach to AR storytelling encourages many stories to be created for Sweetgrass Bear, while recognizing that Indigenous Knowledge Keepers should retain ownership and control of hosted content. Using marker-based AR, users scan the bear sculpture, which triggers an introductory sequence and invites them to select a particular story to hear. Once they have selected a story, they can then scan parts of the bear as AR-markers (e.g. the Cree syllabics on its legs), and view content in AR such as audio, images, and video.

Augmented Reality was defined early-on in this widely cited passage from Azuma (1997):

Augmented reality (AR) is a variation of virtual environments (VE), or virtual reality as it is more commonly called. VE technologies completely immerse a user inside a synthetic environment. While immersed, the user cannot see the real world around him. In contrast, AR allows the user to see the real world, with virtual objects superimposed upon or composited with the real world. Therefore, AR supplements reality, rather than completely replacing it. Ideally, it would appear to the user that the virtual and real objects coexisted in the same space. From: Azuma, R. 1997. “A Survey of Augmented Reality.” Presence: Teleoperators and Virtual Environments 6: 355-85.

Fiducial Markers: images that the computer imaging system can use as points of reference to link the digital and physical realms. They can also be used to generate measurements of the scene [wiki entry]. Many AR approaches use fiducial markers, but whether or not they use the markers to get measurements is unique to the design of the marker and the design of the system. A wide range of image types can be used as markers, from simple black and white pixel grids to detailed magazine advertisements.

Geolocative: Digital content that is tied to a particular geographic location / set of GPS coordinates. AR content may or may not be geolocative, depending on whether it is triggered by GPS coordinates or by scanning an image that could be printed anywhere.

POIs (Points of Interest): These are the real-world locations where AR content exists within geolocation-based AR applications. For instance, in the context of an AR app like Pokemon Go, these were automatically created based on mapping data for where parks and monuments are located. This automated approach failed to account for context, and players of the game would play it in inappropriate locations such as graveyards, since these were also marked as POIs.

SLAM (Simultaneous Localization and Mapping): “the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it” [wiki entry]. In practice, this means an AR app is able to both scan an object in 3d space and determine the position of the phone/user in relation to it, and continuously update this as the phone/user moves through space. This allows AR apps to not require markers, since they just scan objects instead. This is the technology used by self-driving cars to keep track of their location and the environment as they drive. This is typically called “Markerless AR” or simply SLAM by the various AR platforms.

In winter 2018 we completed a survey of major AR platforms to help us determine the possibilities for our project. We present this survey below as a resource for others engaging in this work.

ARCore

ARCore is an SDK released by Google for developing augmented reality apps on certain newer Android phones. It takes advantage of the devices’ motion sensors as well as the cameras. It can track environmental lighting levels to better integrate the AR content, as well as flat surfaces. Given the backing by Google, it is likely to become widely used in the near future.

ARKit

ARKit is an API from Apple for building augmented reality apps, supported on newer iPhones. It takes advantage of the devices’ motion sensors as well as the cameras to allow for SLAM-style AR. It can track both environmental lighting levels and flat surfaces in order to better integrate content. Given the backing by Apple, it has already had significant adoption.

ARToolKit

ARToolKit is the most widely-used open source library for creating Augmented Reality applications, however the project appears to have been recently abandoned as its website has been removed. It has the ability to track square marker patterns and images, overlaying graphics in real time. A version of this library has been developed for javascript, AR.js, allowing web-browser-based AR. As a computer vision library, ARToolKit is not itself a platform and would require an application to be developed using it. It was challenging to find examples of commercial products using it, however it is clear that many people have used it in a research context. It is a very established library, having been around since 1999.

Augment

Augment is a platform that allows marketers to use augmented reality as a sales tool. The platform allows users to upload and manage 3d models of their products, which potential customers can view in AR. This allows online shoppers to see objects in AR before purchasing them. It offers an SDK so that businesses can incorporate links to view AR content within their existing eCommerce platforms. It allows companies to integrate its SDK into custom apps or to simply use the default Augment viewer app. Augment runs on a combination of Vuforia and OpenGL. This platform offers free educational licenses, though its emphasis is squarely on product marketing with objects, not storytelling, so it would be a poor fit for narrative projects.

BlippAR

BlippAR was one of the early companies to develop AR for marketing, specifically for augmenting print publications. It has since expanded to include object recognition and facial recognition, and now urban navigation in their City AR app. It features a Blippbuilder web app allowing non-developers to create content for the platform, however this interface currently does not support 3d assets. There are free educational licenses for using Blippbuilder, allowing unlimited content creation and for an unlimited license duration. Students require a code to view the content, though, and it can only be used for educational purposes, not for marketing the institution. Their cloud-storage of data would likely be a problem for indigenous projects concerned about data ownership.

CraftAR

CraftAR is a suite of tools for developing AR applications. Developers can create apps using their set of image recognition and AR SDKs, plugins and libraries. It has a cloud-based Content Management System and a drag-and-drop web tool for authoring AR experiences. The platform is free to use for developing, and pricing only comes in when a team wants to publish their app; the pricing information is not shown publicly, and there is no indication of educational licenses.

EasyAR

EasyAR is an SDK for creating AR apps. It offers free and paid versions, with the paid version having additional object tracking abilities such as SLAM. Users register on the online platform for obtaining licenses and testing their apps before release. It is based in China and has a stronger foothold in that market, such as the Chinese-only AR version of the game Monument Valley.

HP Reveal [formerly Aurasma]

HP Reveal allows users to create and view AR content. Creators upload the content on their web-based Studio platform, and users can view it either through the proprietary app or through apps that integrate its SDK. The AR content can be animations, videos, images, and 3d models. It features a reasonably user-friendly online CMS for creating AR stories. It is mainly being advertised as a marketing tool, while still acknowledging its potential as an educational tool. Despite using cloud-based content storage, the Terms of Service (in August 2018) made it clear that users retain ownership over their content.

Kudan

Kudan is a computer vision company that offers a framework for developing AR applications. It supports marker tracking as well as a markerless SLAM approach to AR. It is free for education and ‘indie’ users (businesses making under 1 million GBP / year). Kudan works on multiple platforms, and supports integration with various CMS platforms such as Wordpress or Adobe CMS. It comes with a Unity plugin, and has a reasonable amount of public documentation.

Layar

Layar is an AR browser and web-based CMS for creating AR experiences triggered by markers. The platform appears to target print advertising, as it is a subscription service based on the number of pages that a client wants to host, and is quite expensive to use. It includes an optional SDK for integration into custom apps. Layar has been acquired by BlippAR, however it has been maintained as a separate platform.

PixLive

PixLive is an AR viewer and online CMS system. It also offers an SDK for integration in custom apps, and supports locative content through beacons. The content management and authoring systems seems to have a lot more functionality than other easy-to-use AR platforms, however PixLive is quite expensive compared to alternatives that offer free educational versions, and the cloud data storage of the content could be a problem in the context of indigenous data ownership concerns.

Vuforia

Vuforia is a widely-used SDK for developing AR experiences. It supports a wide range of platforms, includes Unity integration, and features a wider-than-normal set of marker types including 3d objects, cylinders (e.g. drink bottles), and proprietary barcodes (fiducial markers). It does support markerless targets but not using SLAM. Unfortunately the free version only supports applications that place an object in a space (without using markers). The cheapest marker version is a $500 one-time product key, and there are no educational licenses. Targets are managed on the cloud, however the actual data that is displayed in AR apps appears to be hosted outside of their platform.

Wikitude

Wikitude is a feature-rich AR SDK, including the ability to create AR experiences based on location instead of markers or SLAM. It is one of the more established players in AR, and is more stable as a result. It includes a Wikitude Studio content creator web app for users to create AR experiences if they aren’t making a custom app. There is a free educational license, however teams need to apply for it, and it there are restrictions on distribution and the time period it lasts for that are likely too restrictive for most projects. The pricing for commercial licenses is not publicly displayed.