GNM_RCS_Web_Content_Interfaces_TS

Guardian News & Media

GNM RCS

Web content interfaces

Technical specification

Prepared by O3 Team Limited

Authors Nigel Robson

Creation date 11/12/2013

Document Ref. GNM_RCS_Web_Content_Interfaces_TS.docx

Version draft for review

1. .Introduction
  1. Purpose

The document GNM_RCS_System_Interfaces_FS.docx is the functional specification that describes what business functions RCS supports in relation to its interfaces with other GNM systems as well as any external integration.

This document is one of a set of technical specifications that provide details of how those functions are implemented in RCS.

1. Scope

This document focusses on the 2 interfaces RCS has to find out about content published on the website: the R2 interface which is currently in use; and the API interface which is set to replace the R2 interface. Separate documents deal with all other inter-system interfaces that pass data between RCS/SLM and other systems.

This document is intended as a high-level technical document outlining how the relevant business functions are implemented in terms of software modules.

Importantly, this document does not aim to provide the level of detail that would be required in a programming specification in areas such as program structure, detailed business rules, data integrity, validation, locking considerations, data security, and calls to/from other software modules, performance considerations, and so forth.

For details of program logic and coding, the reader should refer to the program files themselves.

1. .Web content interfaces
  1. R2 interface

The R2 interface is the original interface that has been in use from June 2009 and remains in use at the time of writing. It is about to be phased out, and replaced by the API interface which is described in the next section of this document.

1. Data model

The data model that supports the R2 feed is quite simple. For each item of website content there can be multiple tags, multiple keywords, and multiple contributors.

These tables exist in the TLIB schema to be consistent with the logging of print content in that schema. RCS extracts the data it requires from these tables.

1. XML feed

This interface is an XML feed that RCS requests from R2, and is access using this URL (with parameters o identify the time period required): http://cms.guprod.gnl/tools/external/rcs/list

Excerpts from this feed, showing one story and one image by way of example, is shown below:

<contentType>Article</contentType>

<section>Society</section>

<title>Thousands of patients at risk from incorrect use of IV drips, warns Nice</title>

<url>http://www.theguardian.com/society/2013/dec/10/nice-warning-risk-patients-hospitals-ignorance-intravenous-drips</url>

<description><p>National Institute for Health and Care Excellence issues guidelines for medical practitioners on correct dosage and timing</p></description>

<publicationName>The Guardian</publicationName>

<byline>Haroon Siddique</byline>

<string>Hospitals</string>

<string>Health</string>

<string>Medical research</string>

<string>Science</string>

<string>The National Institute for Health and Care Excellence (Nice)</string>

<string>Society</string>

<string>Doctors</string>

</keywords>

<name>Haroon Siddique</name>

</contributor>

</contributors>

</rcsContent>

<pictureContext>Trail Picture</pictureContext>

<caption>A nurse checks a drip before surgery. Surveys suggest 90% of doctors do not know the correct dosage and timing for IV drips. Photograph: Richard Nowitz/Corbis / A nurse checks a drip before surgery. </caption>

<url>http://static.guim.co.uk/sys-images/Guardian/Pix/GU_front_gifs/2013/12/9/1386618076160/A-nurse-checks-a-drip-bef-004.jpg</url>

<source>Corbis</source>

<photographer>Richard Nowitz</photographer>

<parentUrl>http://www.theguardian.com/society/2013/dec/10/nice-warning-risk-patients-hospitals-ignorance-intravenous-drips</parentUrl>

</rcsPicture>

1. Parsing software

The software that parses this feed is quite complicated and is not discussed in detail here but the technique is described in the RCS Software Techniques technical reference document

1. Website content to attribute screen

The screen below shows content published on the website that has no contributor tags. These items are queued for the RCS Administrator to attribute to one or more contributors.

The screen, called Website content to attribute, is accessed from the menu option: Content → Web content to attribute (old feed). The Oracle Form that is opened is rcs_wcon_010_pc.fmb

The user can disregard content from this screen, associate it with one of more RCS contributors, or choose to “Ignore” it..

1. API interface

The API interface is still under development at the time of writing. The move to this interface is planned for the following reasons:

- It is a standard API already maintained for many other clients; and
- It includes far more content and more detail about that content than the R2 feed.

1. Data model

The data model that supports the API feed is more complicated than for the R2 feed. It is designed to match the structure of the feed although this does mean there is some duplication of data.

These tables exist in the TLIB schema to be consistent with the logging of print content in that schema. RCS then extracts the data it requires from these tables.

1. Content items and relationships

The content received through the feed has the following types:

- Article
- Audio
- Audio slideshow
- Cartoon
- Competition
- Crossword
- Data
- Document
- Gallery
- Graphic
- Interactive
- LiveBlog
- Picture
- Podcast
- Poll
- Quiz
- Signup
- Sudoku
- Table
- Trailblock
- Video

The content stored in the WEB_API_CONTENT table can be thought out as standalone content e.g. a video, or top level content with other content embedded.

Top level content with embedded content could be an article with embedded images, or a gallery with embedded images etc These embedded items are content in their own right that must be rights managed, but are only defined with then ELEMS and ASSETS tables. In then case of images the ELEMS and ASSETS tables may record the many different sizes of each image, though RCS only needs to process one of them. But if multiple images are embedded or a different thumbnail linked to the main article then RCS needs to process each of them.

1. The feed

This interface is an XML feed that RCS requests from the production systems.

The feed is accessed from the following URL with parameters passed that define the period being requested:

http://prod-mq-elb.content.guardianapis.com/api/search.xml?

An excerpt from the feed is shown below, for a single item, as an example for the formatting:

David Tennant should have foreseen his wife's fate but Mitchell and Webb manage to avoid the obvious, writes <strong>Andrew Anthony</strong>

</field>

Rewind TV: The Escape Artist; Ambassadors; A Very English Education – review

</field>

<field name="has-story-package">false</field>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154805650/Designer-life-David-Tenna-004.jpg

</field>

David Tennant should have foreseen his wife's fate but Mitchell and Webb manage to avoid the obvious

</field>

http://static.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154805650/Designer-life-David-Tenna-004.jpg

</field>

<field name="allow-ugc">false</field>

<field name="is-premoderated">false</field>

<field name="byline">Andrew Anthony</field>

<field name="publication">The Observer</field>

<field name="live-blogging-now">false</field>

</fields>

<tags>

<tag web-url="http://www.theguardian.com/profile/andrewanthony" type="contributor" web-title="Andrew Anthony" api-url="http://content.guardianapis.com/profile/andrewanthony"id="profile/andrewanthony" bio="<p>Andrew Anthony has been writing for the Observer since 1993 and for the Guardian since 1990. He is the author of On Penalties, published by Yellow Jersey Press and The Fallout, published by Jonathan Cape</p>" byline-image-url="http://static.guim.co.uk/sys-images/Guardian/Pix/contributor/2007/09/26/andrew_anthony_140x140.jpg" rcs-id="GNL385518" r2-contributor-id="15352"/>

</tags>

<media-assets>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154801279/Designer-life-David-Tenna-001.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154803049/Designer-life-David-Tenna-002.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154804371/Designer-life-David-Tenna-003.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154805650/Designer-life-David-Tenna-004.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154806887/Designer-life-David-Tenna-005.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154808179/Designer-life-David-Tenna-006.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154809408/Designer-life-David-Tenna-007.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154810646/Designer-life-David-Tenna-008.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154811952/Designer-life-David-Tenna-009.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154813226/Designer-life-David-Tenna-010.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154811952/Designer-life-David-Tenna-009.jpg

</field>

<field name="source">BBC/Endor Productions</field>

<field name="photographer">Steffan Hill</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist.

</field>

<field name="credit">Steffan Hill/BBC/Endor Productions</field>

Designer life: David Tennant and Ashley Jensen as the charmed couple in The Escape Artist. Photograph: Steffan Hill/BBC/Endor Productions

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383153749328/Unpredictable-David-Mitch-001.jpg

</field>

<field name="photographer">Colin Hutton</field>

Unpredictable: David Mitchell and Robert Webb in Ambassadors.

</field>

<field name="credit">Colin Hutton/BBC/Big Talk</field>

Unpredictable: David Mitchell and Robert Webb in Ambassadors. Photograph: Colin Hutton/BBC/Big Talk

</field>

</fields>

</asset>

https://static-secure.guim.co.uk/sys-images/Guardian/Pix/pictures/2013/10/30/1383154157908/A-Very-English-Education-001.jpg

</field>

<field name="photographer">Patrick Smith</field>

<field name="alt-text">A Very English Education</field>

<field name="credit">Patrick Smith/BBC</field>

Alma mater: Donald Payne, one of the old boys in A Very English Education. Photograph: Patrick Smith/BBC

</field>

</fields>

</asset>

</media-assets>

</asset>

</asset>

</asset>

</asset>

</asset>

</asset>

</asset>

</asset>

</asset>

</asset>

</asset>

</assets>

</element>

</asset>

</assets>

</element>

</asset>

</assets>

</element>

</asset>

</assets>

</element>

</elements>

</content>

</results>

</response>

1. Parsing software

The software that parses this feed and stores the data in the database is very complicated and is not discussed in detail here. However the technique is described in the RCS Software Techniques technical reference document

1. PicDar URNs

In order to be able to link images published in print with images published on the website the PicDar URN is needed. However, with this feed th PicDar URN is only available if the content has been produced using Composer. For all other images the URN is missing. The ensure RCS gets these URNs an in-line process accesses the Octopus database to try to gather this data, and any IPTC header information that is available.

The availability of the Octopus database is critical to this process working.

1. Website content to attribute screen

This screen performs the same function as described for the R2 feed screen, but processes the API feed data instead of the R2 data. The screen, called Website content to attribute to contributors, is accessed from the menu option: Content → Web content to attribute

The Oracle Form that is opened is rcs_waco_010_pc.fmb

The order in which the “Fields” data is displayed is configurable (see next section). The current configuration has the wordcount appear at the top of the list of fields.

1. Assets

The assets associated with an item of content can be viewed by clicking on the minimized window labelled “Assets”:

1. Elements and their assets

The elements, and their link to the assets are visible in another minimized window labelled “Elements”.

Whilst there is a certain amount of duplication with the Assets window, each window does offer different data.

The ID of the image may contain the CMS ID of the content: if it is in the form “gu-image-nnnnnnnn” then the “nnnnnnnn” is the CMS ID of the image. This ID can then be used to access the Octopus data to get the PicDar URN if that is missing.

1. Field configuration for the API feed

The API feed can be configured so that some fields in the feed can be ignored whilst others are processed and stored in RCS.

The screen, called Website fields available for extraction, is accessed from the menu option: Content → Web fields to extract

The Oracle Form that is opened is rcs_wafi_010_pc.fmb

The “Maps to” column is used as an aid memoire to identify the data in a way better understood in the Rights team.

The “Include?” checkbox can be toggled to determine whether the field will be processed or not.

End of Document

Keywords (or tags) are important to provide accurate search results. They are vital if you have attached rather than pasted content to this page.

Page updated

Report abuse