Moving from traditional desktop publishing to distributed, Wiki-based collaborative authoring and publishing is a realistic and achievable goal. The business case for making such a transition is also compelling— a 500-user commercial Wiki license is half the cost of a single-user FrameMaker and WebWorks license.
This paper describes a software documentation group’s transition from a highly optimized FrameMaker and WebWorks Publisher-based publishing process to a Wiki-based replacement. It goes through the steps the group took to arrive at their solutions and lists the sources they used to learn about their options.
This paper also touches upon the quality improvements—one of the benefits of collaboration— that motivated the team to work on crafting an open solution.
Agilent EEsof EDA is a software Division in Agilent Technologies. Over the years, EEsof’s development activity has grown from an annual release of three software products to a quarterly release of eight software products. For the documentation team working on the online help and manuals, this growth in development activity—from one major release per year for three products, to 3-4 releases for eight products—without any corresponding increase in people, has led to an unrealistic workload of 200-plus books (30000+ pages) updated multiple times a year.
Original EEsof Publishing Workflow
In response to these changes, the EEsof documentation team crafted an elaborate system to document software updates, integrate the published content into the software build, and update the information on the EEsof website. The documentation team also implemented a robust content versioning mechanism to enable them to recreate any arbitrary version of the content source and published output.
This ability to update, single-source, and publish a high-quality product, however, came largely under the yoke of rigorous processes. Updating content required a high level of desktop publishing expertise and process-specific training. Not many people knew how to update the documentation and fixing even the simplest formatting or spelling error involved numerous steps. It could take months before the few who knew how to make changes, found the time to fix and update the build and Web so that users could see it.
Existing Update Timeline and Workflow
In addition, even when subject-matter experts were trained to work with documentation files, standardizing on the version of tools to be used turned out to be more complicated and expensive than expected. For example, FrameMaker 6 and WebWorks Professional—the versions used in the United States—are no longer available in Beijing and Belgium. Changing the processes to work with updated versions would require extensive effort even if EEsof could justify the time and expense of upgrading all its licenses.
The problem statements defined by the documentation group were as follows.
Documentation had become a developmentbottleneck
Tools and processes were complicated, time-consuming, and resource-intensive
Activities weren’t focused on content creation
Content updates had become infrequent
When the EEsof documentation team began looking for cost-effective alternatives to merely upgrading to newer versions of their software (FrameMaker and WebWorks Publisher), they noticed that the Wikipedia website contains over two million articles in English, with over 100,000 articles in each of 17 other languages, mostly managed by people who do this in their spare time. Given the astonishing collaboration occurring on the Wikipedia, the immediacy of viewing the updated content online, and the quality, depth, and breadth of the content that is available as a result, the EEsof documentation team decided to investigate the Wiki as a collaborative authoring and publishing tool.
Desired Update Timeline and Workflow
While the use of Wikis as collaborative authoring systems is well-established, and the use of templates and template engines to deliver content in various formats is a well-documented theoretical concept, the EEsof documentation team was unable to find a single example of a Wiki used as an end-to-end authoring and publishing solution.
The closest the team came to an analogous use was an instance of online help pointing to a Web-based location instead of a user’s local drive and the use of a wiki to create an online knowledge base.
Given the obvious availability of all the standards and technologies required to make it happen, especially the translator tools to convert content into any desired output format, the EEsof documentation team saw this lack of existing examples as an opportunity to come up with their own solution.
Starting with DMOZ, the Open Source Directory website, the EEsof documentation team undertook an informal investigation of the world of Wiki engines. Armed with background information and links to other resources gleaned from this site, the team then proceeded to experiment with the “What-If” simulations at the Wiki Matrix website to understand how their choices of technology and features affected the list of Wiki engine possibilities.
WikiMatrix (http://www.wikimatrix.org/)
The Wiki Matrix website also provided detailed information on each engine, along with links to other websites and online discussion groups. The EEsof documentation team also used the Wiki Engines website to continually verify that expertise to work with the technologies used by the various possible Wiki engine candidates was available within the team.
Once the EEsof documentation team had identified three possible Wiki engine candidates, based upon feature comparisons, they began the task of installing and testing each one of them.
The test harness consisted of thousands of html files generated from books in five different categories (Getting Started, User Guide, Theory, Reference, and Catalog) that were ported into each Wiki installation to test the ease of use and capabilities of the engines in the following areas.
Migration
Authoring
Publishing
Content Management
Access Privileges and Control
The following Wiki engines were tested.
MediaWiki, the open source wiki engine that powers the Wikipedia
TWiki, an open source wiki engine for the Enterprise solution
Confluence, a commercial Enterprise wiki engine
Choosing one engine over another proved to be a more daunting task than expected, because each Wiki engine has strengths that could well lead to it being chosen.
The strongest contender, obviously, was the MediaWiki, given its Wikipedia pedigree, but the team discovered a few drawbacks:
The technical expertise required to install and maintain it.
The effort it would take to configure permissions to remove content and modify article titles.
The complexity of using it with an Oracle database.
While these may not even be drawbacks, given a technically more adept team, and different circumstances, for the EEsof documentation team they were daunting tasks.
With TWiki, the decision was simplified by its speed, responsiveness, and scalability issues. Again, these issues need to be viewed as a reflection of the team’s lack of expertise rather than the tool’s limitations.
Eventually, the Confluence wiki engine prevailed on the strength of a few attributes—its commercial support, ease of installation and configuration, and its user-friendly interface.
Confluence Wiki (http://www.atlassian.com/)
Once the Confluence wiki engine was chosen, the task of moving the content into its database was broken up into a number of steps.
Migration Workflow
The translation was accomplished in two stages. In the first stage, the WebWorks Publisher template was modified to generate text files in a format as close to the Wiki’s native markup language as possible. In the second stage, these quasi-markup files were modified using Perl scripts that had been written during the testing phase. These Perl scripts cleaned-up and added markup that wasn’t taken generated by the WebWorks Publisher template.
Translation Scripts
With Confluence, the content transfer was also simplified by its support of the WebDAV protocol (Web-Based Distributed Authoring and Versioning), which was used to transfer files directly into the Wiki.
Perhaps a majority of the time and effort was spent in editing and cleaning-up the transferred content. Some of this was accomplished using Perl scripts that searched and replaced pervasive issues with a detectable pattern, such as an extra space before and after punctuation. The rest (or most) of the time was spent on finding and correcting a small percentage of the content, which required manual copy-editing. Even though less than 2% of the content required manual editing after translation, finding it involved manually sifting through thousands of pages.
Daunting as it sounds, copy-editing all the content to verify its integrity provided an excellent immersion into the nuances of this tool’s usage. The time spent in gaining this expertise during migration also resulted in savings of as many, if not many more, hours of editing time during actual deployment. The expertise gleaned from this exercise has made the documentation team experts in the tool, and the transition from FrameMaker to Confluence has not been accompanied by any loss of “expert” status for the writers. This, in turn, has contributed to an accelerated and enthusiastic adoption and evangelizing of the tool.
Once the content had been edited, it was connected by adding back hyperlinks, table of contents, menu-lists, and front-end pages.
Privilege and access were some of the most intensely debated topics as the EEsof documentation team began setting up an infrastructure the other development teams could use. Questions on how much access to provide and what privileges to enable were, and remain to this day, only partially answered.
Eventually, the team made the leap of faith, as much as is possible in a commercial organization, to adopt the open philosophy of the Wikipedia. When confronted with a choice, the team has leaned towards not restricting the content editing privileges of its users, the larger development team of subject-matter experts.
Wiki Groups and Privileges
As a result, the number of groups has been kept to a minimum, and the access privileges of each group have been broadly and simply defined. The guiding philosophy has been one of openness that begins with the assumption that for people to work in a truly collaborative environment, everyone can and needs to be empowered to make content changes as they see fit.
The open solution goal was also combined with a minimalist interface to arrive at an easy-to-use and uncluttered authoring interface, which can only be changed by the documentation group.
Wiki Authoring Interface
Despite the pressure and temptation to add more features and functionality, the EEsof documentation team has kept the interface elements to the bare minimum. In addition, the interface is set up to display items conditionally—if a person lacks Edit privileges, the menus are not displayed.
Limiting the interface and the choices offered to the user in this manner has led to a universal acknowledgment of a vastly "usable" interface. Subject-matter experts across the organization, the ultimate end-users, have embraced this new interface without any negative feedback to date.
The ability to edit content at this initial stage of public access, though, is enabled only for content creators within EEsof; end-users can see but not edit content. As things stand, edit privileges for end-users is going to be phased in with the ability to first comment and provide feedback.
The EEsof documentation team also put together a comprehensive training course and offered it to small groups as part of the rollout of their Wiki-based authoring system. Before offering these training sessions, one-on-one training was provided to early adopters in each group. Recruiting and training early adopters before running the training sessions ensured an in-house “expert” that other members of each group could turn to. It also provided early feedback that was used to modify the training sessions to fit the audience needs and address their specific concerns.
The ten, team-based training sessions conducted were followed-up with open, make-up sessions.
Using a combination of guided tours and live demonstrations, small groups of subject-matter experts all over the world were introduced to the Wiki-based authoring system. This training was followed up with open Q&A sessions and individual help, as needed.
Wiki Training and Quick Tour
One of the most telling results of this project is reflected in the terminology used by people across the organization. From a "Learning Products Wiki," which is how it was termed at the beginning, it is now called "the Wiki." The spikes in page viewing activity—another monitoring tool built into the Wiki—already correspond to periods of intense development activity. In other words, people have quickly incorporated looking at and editing in the Wiki into their development routine.
In terms of specific project objectives, the following goals have been accomplished with the transition to the Wiki as an authoring, publishing, and content management tool.
Project
collaborative authoring
unobtrusive moderation
on-demand publishing
Business
capacity–low barrier for content creation
turnaround—months to days
User experience
ease of use
faster display
frequent updates
Documentation no longer holds up the software development process. In addition, all subject-matter experts now have the requisite access and training to modify content at any time. As various part of the organization—R&D, Technical Support, and Product Marketing—interact with the end-users, they have the ability to update content at any time, as soon as they discover the need to do so.
For the documentation team, the key statistic is the number of people actively editing in the Wiki. The quality of the documentation improves as the number of people and the number of interactions increase. In other words, the information gets better as more people look through it and refine it.
The first identified problem is no longer an issue:
Documentation is no longer a development bottleneck
Increasing Collaboration
Adding and editing content is simple because the documentation process now uses the tools and conventions that subject-matter experts are already familiar with. The Confluence Wiki uses a Rich Text editor with an interface that incorporates the most commonly used buttons in its formatting toolbar.
Simplified Formatting Interface
In addition to the ease of formatting and entering content, most of the review and publishing tasks have either become transparent to the users or have been simplified and automated. As a result, a subject-matter expert does not need a high level of desktop publishing expertise or process-specific training.
Simplified Processes
The second identified problem has been resolved for the most part:
Processes are no longer complicated, time-consuming, or resource-intensive
A majority of the EEsof documentation team’s time is now spent validating, moderating, and improving content. Instead of focusing their efforts on ensuring content is intricately and exactly formatted so that it can be published in a professional manner, the team focuses on ensuring the information is valid, helpful, and accessible.
Page Views in the Last 80 Days
In the first few weeks of its implementation, 20+ R&D engineers used the Confluence Wiki to create and update as many pages as four extra writers that we don’t have. In addition the documentation team was able to monitor and moderate the content as it was being created instead of having to wait until the very last minute to see, let alone edit their inputs.
The third identified problem has been addressed as well:
Activities are now focused on content creation
Content management in a Wiki is built-in and automatic. Content update on the Web is not the result of a laborious task performed as an afterthought. It is an instantaneous and automatic consequence of a process that taps into the immediacy of the Web. In addition, a number of reporting mechanisms that are built into the Confluence Wiki provide activity reports on an ongoing basis. By monitoring these reports, and in some cases subscribing to them, writers are able to ensure the quality and integrity of the content even as it is being created.
Each time a topic is edited, a writer gets an RSS feed informing them of the change. At the end of the day, the Wiki also generates a daily summary of pages changed. These change notifications include links to the page that was changed. The changed pages provide the ability to view the changes and even revert to the previous version of the content, if needed.
Content Management Dashboard
The fourth identified problem is no longer an issue with instant updates:
Content updates are no longer infrequent
To meet its constantly growing workload, the EEsof documentation team could have taken the safe and traditional route of upgrading to newer versions of its desktop publishing tools. This seemed the logical choice given that the use of these tools forms the core tool expertise of the group.
By choosing to craft a more accessible Wiki-based solution, the team has realized the benefits of collaborative authoring and simplified content management. In addition, it has discovered that the migration can be accomplished with relative ease, even with the limited technical skills available in a traditional documentation group.