Open University Case Study | Technical Perspective
About The Open UniversityThe problem space
Baseline position
Description of activities: OCCAM project
Description of activities: XCRI-CAP 1.1 implementation
Issues
XCRI-CAP in practice
About The Open University
The Open University (OU) is the United Kingdom's only university dedicated to distance learning. It is the largest university in the UK by number of students, and its course offerings are correspondingly large and complex. It has around 150,000 undergraduate and more than 30,000 postgraduate students, 11,000 of the latter studying for Higher Degrees.
The OU's style of teaching is called ‘supported open learning’. Nearly all students are studying part-time. About 70 per cent of undergraduate students are in full-time employment. More than 50,000 students are sponsored by their employers for their studies. A third of the OU’s UK undergraduate students have entry qualifications lower than those normally demanded by other UK universities.
Most OU courses are available throughout Europe. Some of them are available in many other parts of the world. More than 25,000 OU students live outside the UK.
top of pageThe problem space
The purpose of the OU’s initial XCRI-CAP 1.0 implementation was to extend, enhance and standardise the existing technical methods through which the OU supplied its course advertising data to third party aggregators, both to make this process more efficient and to provide the wider community with a model of good practice using the XCRI Course Advertising Profile (CAP) schema. This purpose was achieved through the JISC-funded XCRI mini-project ‘Open Course Collection and Aggregation Model (OCCAM)’. The success of the OCCAM project encouraged the university to extend the work to a fully live feed using XCRI-CAP 1.1, so that third parties could carry out fully automated harvesting of the OU’s course marketing data.
Technical
The primary driver for change was to improve the effectiveness and efficiency of the supply of course marketing information to third party data aggregation organisations, including at that time:
The National Learning Directory (via UCAS), now the SFA’s Course Directory Learndirect Scotland
Learndirect Northern Ireland
Graduate Prospects
North West Workplace
ECCTIS
North East Coursefinder
The technical context of the OU’s XCRI implementation in 2006 was complex. Course marketing data was collected together from several different sources into a bespoke database (‘P12’), which was used as a means of generating pages for the OU’s main website. This database was the only comprehensive and up-to-date centralised source of authoritative course marketing data apart from the website itself. As it was already being used by APS Ltd for supply to external organisations, it was the obvious immediate candidate for sourcing data for the XCRI feed. However, the team recognised that an existing project to implement a Content Management System (CMS) as a new source of data for the website was in existence and was expected to become the new authoritative source in a time scale of two or three in the future.
The P12 database included a significant quantity of HTML marked up data and required manipulation and re-formatting by APS to make it suitable for delivery to third parties. APS had a suite of software to carry out these tasks. Any new system would require redevelopment of this suite.
top of pageBaseline position
Existing processes were modelled using Unified Modelling Language (UML) diagrams. The purpose of the modelling exercise was to aid the project team’s understanding of existing processes, so that we could confirm intervention points and actions for development. While we intended to carry out a comprehensive business process view, we found that it was sufficient to create only the Process Behaviour and Requirements Views. These gave a coherent overview of the current requirements and information flows in relation to courses information delivered to data collecting organisations.
Description of activities: OCCAM project
For the Occam Project we were using XCRI-CAP 1.0, and the project ran from March to September 2007.
The main technical activities centred around mapping the OU course data structures to XCRI-CAP, creating a mechanism to convert the original data to XCRI-CAP and the development of the web service to output the XCRI-CAP.
Mapping
There were two main issues that we had to address in relation to the mapping of the data.
A structural issue was that OU provision had traditionally been divided into two main types: courses that led to the award of credit, and programmes that led to the award of qualifications. The former were structurally equivalent to modules or units at other universities, but were the primary means through which the OU articulated its offerings – potential applicants would select from a list of these courses, and then decide to apply for a particular presentation of that course. However, in addition the OU marketed ‘qualifications’ or coherent routes to awards, which enabled the display of OU provision in a similar fashion to traditional universities, even though enrolment was via courses and not programmes. There was potential confusion between the OU’s term ‘qualification’, meaning a route to an award, and the more conventional use of the term ‘qualification’, to mean the award itself. In essence the OU has two types of XCRI-CAP ‘course’ element: a course and a qualification. However, while the OU course has presentations, the OU qualification does not – it is a creation in order to permit comparison with provision from other universities. To overcome this problem, OU course data for third parties has historically contained course records of these two types, together with dummy presentations for the qualification records.
These structures were mirrored in our mapping from the existing OU data structures to XCRI-CAP.
The second difficulty was that the OU data set does not carry an audit trail or a date of last update. The reason for this is that changes are not logged comprehensively in the originating sources, and this is not a requirement for the website, which is refreshed completely each time it is updated. We had to provide OU data as whole data sets in the feed, which means that tracking of additions and deletions has to be done as an additional process when the feed is consumed. This was recognised as a difficulty to be addressed at a later date.
Mapping was carried out using a combination of an Excel spreadsheet and the Altova commercial mapping tool, MapForce. The MapForce software has a ‘drag-and-drop’ interface both from and to XML instance documents that meet a DTD or schema, together with XPATH functionality for transformation of nodes of the source data. Source and target files can be XML files, databases or comma-separated value files. In addition to a graphical view of the mapping and transformed data, MapForce can produce XSLT, Java and other programming language outputs, so that transformations can be run independently of the Altova product. A further advantage is that it does produce real time outputs which can be reviewed and validated as part of a sequence of iterations.
The mapping was created using many trial iterations. Output XML files were validated periodically against the XCRI-CAP schema, in order to check the accuracy of the mapping. Altova XMLSpy was the chosen tool for validation. Where particularly complex transformations were required, the mapping was broken into stages, each having a separate MapForce mapping. This process meant that the mapping was clearer visually, and PC performance limits were avoided.
Web service
A SOAP web service to enable the harvesting of the data was designed, and then eveloped and implemented using the .Net framework. The web service can be viewed at: http://www3.open.ac.uk/occam/courseinfo.asmx. The web service was initially trialled using test data and subsequently implemented using the full OU course catalogue for August 2007 in XCRI-CAP format.
Owing to the need to continue to provide additions and modifications to the data, the web service used an XML instance file provided by APS as the back end, until such time as a CMS driven back end could be developed.
The file size of the full course catalogue was nearly 5 Mb, which presented a potential performance problem. Recognising that the main usage might be an infrequent but regular harvesting of the whole data set, the project decided that publishing as a single XML data file was sufficient.
top of pageDescription of activities: XCRI-CAP 1.1 implementation
The OU decided to upgrade the web service to XCRI-CAP 1.1 in tandem with the migration to the new CMS. The development work for this was handled almost entirely by OU developers, with APS involvement limited to initial mapping of OU data to XCRI-CAP 1.1, and testing and validation of outputs from April to September 2009. The process was straightforward and only required a couple of iterations to complete successfully.
The significance of this work was that the back end source for the web service was the new CMS, so there was a direct connection to the other parts of the OU’s infrastructure. This web service was offered in a RESTful architecture, rather than SOAP, so that it was easier to develop, maintain and consume. We operated both the original and the new service in tandem for updates in June and August 2009, in order to check processes and outputs thoroughly. The XCRI-CAP 1.1 ‘native’ service from the OU was launched in August 2009.
Part of the role of APS in relation to supply of OU data to third parties is to add specific coding schemes, classifications and controlled text fields to the original data, so that it can be used much more easily by other organisations. APS has therefore developed a ‘value added’ XCRI-CAP 1.1 SOAP web service feed on behalf of the OU, which has this extra material in it.
top of pageIssues
The OCCAM project highlighted some technical issues in XCRI-CAP implementation, which are listed here:
- Security: For the Occam Project we envisaged that the group of users would be strictly controlled, that the pilot version would not require extensive security, and that security would be re-examined on full implementation. For the project, accesses to the web service were logged, but no security system was enacted. For XCRI-CAP web service implementations, each provider will need to decide on an appropriate security model, so that access to the data is available to authorised users only.
- Contextualisation of data: Should information be presented in a generic descriptive format for immediate readability by end users, or broken down into component parts and presented in a form suitable for re-use by computer systems.
- Address format: There is no generally accepted standard format for addresses; it will be important for XCRI-CAP to be able to support emerging standards for both geo-location, which may be used by data collection organisations for searching, and for postal communication.
- Link to wider initiatives, for example MIAP: Communication with MIAP and other initiatives will help to ensure that XCRI-CAP can be integrated with other information systems.
- Vocabularies: The project decided that it would not publish OU vocabularies or validate against them within the XCRI-CAP process. These vocabularies are generally proprietary and would not be used by external agencies, so the team preferred to publish the information in a human-readable form. The use of vocabularies in conjunction with XCRI-CAP was regarded as vital for transformation purposes, and the project recommended that vocabularies are kept separate from XCRI-CAP, but are referenced from it.
- Identifiers: The use of identifiers was likely to be a key issue for the wide adoption of XCRI-CAP. It would be important to be able to label an identifier, so that its context was recorded. In addition the use of persistent identifiers (for example persistent URIs) was not yet wide-spread; specific action to encourage this should be considered.
- Supported open and flexible modes of learning versus ‘traditional’ face-to-face teaching: A major outcome of the project was to suggest the reduction in the number of mandatory elements in the model. Many elements were indicated as mandatory in version 1.0, for example qualification, start date, venue address, that were not relevant for the types of course that are offered by the OU.
XCRI-CAP in practice
The OU XCRI-CAP feeds have been available now for over two years. It migrated to XCRI-CAP 1.1 in 2009, so there has been a significant length of time for their operation. In addition the ‘value added’ feed has been divided into Postgraduate, Undergraduate and All sections, for ease of use.
A further issue now of relevance is how organisations discover the existence of XCRI-CAP feeds. At present there is no nationally agreed protocol for the location of feeds on institutions’ websites, and the main register is a human-readable page of feeds listed on the XCRI wiki. If a protocol for location of feeds could be agreed, then the OU feeds could be readily used through automatic discovery.
top of page
