XCRI Readiness
In addition to the XCRI Self-assessment Framework and the Guide to Implementation, this section of the Knowledge Base summarises the method used by the XCRI Implementation Models (XIM) project for reviewing a learning provider's readiness for XCRI implementation. | Next >> |
Using part of Capability Maturity Model Integration the XCRI Implementation Models project developed a method of characterising an organisation’s management of course advertising information, and describing its approach to processing and publication of this information through assessing the organisation’s position on a range of scales, covering technical, process, information management, data analysis, resourcing, relevant expertise and other variables. The set of attributes formed an assessment of the organisation’s “XCRI readiness” based on previous experience. It provided a framework for identifying problems and associated solutions.
ID |
Attribute |
Value |
Start |
0 |
1 |
2 |
3 |
4 |
End |
1 | Number of courses | Very Large | Very small | ||||||
2 | Complexity of provision | Highly Varied | Simple | ||||||
3 | Number of sources | Multiple | Single | ||||||
4 | Quality of authoring | Poor | Excellent | ||||||
5 | Data structures | Poorly defined | Well defined | ||||||
6 | Update frequency | Less than once/year | Continuous | ||||||
7 | Audit trail | None | Detailed | ||||||
9a | Centralisation (UG) | Mostly decentralised | Complete | ||||||
9b | Centralisation (PG) | Mostly decentralised | Complete | ||||||
10 | Process capability | Ad hoc | Well defined | ||||||
11 | Technical capability | Low | High | ||||||
12 | Organisational context | Hostile | Benign | ||||||
13 | Resources | Minimal | Plenty |
TABLE 1: XCRI READINESS CHARACTERISTICS
The scales of characteristics are designed to be similar but not identical to Capability Maturity Model Integration. The scales run from 0 to 4, where 4 is perfect and 0 is least favourable. The normal range of values was between 0 and 3, with relatively few values at 3.
For each attribute a judgement has to be made as to where the organisation sits in relation to previous judgements. As the scales were operated by one person during the XIM project, a consistent approach was possible. Definitions are given here for each value in each scale.
ID |
Attribute |
Definition of scale values |
||||
0 |
1 |
2 |
3 |
4 |
||
1 | Number of courses | > 500 | 200 - 499 | 50 - 199 | 10 - 49 | < 10 |
2 | Complexity of provision | Large, complex programmes with extensive choice of routes through them. Complex rules on choice. | Wide range of provision across FT, PT, DL, CPD, UG, PG including combined programmes | Some complex types (eg combined degrees, FT/PT/CPD) | Handful of simple types | Very simple single type |
3 | Number of sources | An unknown number of multiple sources | Many sources, but the vast majority are known | > 3 sources, all known | 2 or 3 sources | 1 authoritative source |
4 | Quality of authoring | Poor; quality control is severely lacking; only the very basic data is correct; much of the information is input in an ad hoc fashion. | Average; many items are correct, but some of the data lacks depth, is out-of-date or is incorrect; there are significant gaps. | Good; items are generally correct, but some are lacking in depth and there are acknowledged errors | Very good; items contain correct information with few errors | Excellent; all items correctly completed |
5 | Data structures | Poor; data is held in a non-normalised form with no or poor documentation; additional data structures have been added reactively without sufficient regard to existing ones. | Average; data structures are complex and not internally consistent; data is not fully normalised; the system is not well-documented. | Good; data structures are coherent, but not fully normalised; definitions can be derived from the data. Some links between data items are not well-documented. | Very good; data structures are coherent and most data items defined and documented | Perfect; fully normalised; all data items well defined and documented; vocabularies are accessed through look-up facilities and are managed. |
6 | Update frequency | Less than once/year | Annually | Termly | Approximately monthly | Continuous |
7 | Audit trail | None | Date of new records is recorded. Major updates are recorded by flags but not dates. | Dates are recorded whenever a course or course instance record is updated. | Audit trail is implemented at record level for every significant record type (course, presentation, qualification, venue, etc) | Every update of each significant field is stored, trackable and reversible. |
9a | Centralisation (UG) | UG information sources are decentralised, typically to individual departments | A small proportion of UG data is centralised, most is decentralised. | A significant proportion of the UG data is centralised and the amount is increasing. | Only a small amount of the UG data is not centralised (eg funding data) | All UG course data is held on 1 central source |
9b | Centralisation (PG) | PG information sources are decentralised, typically to individual departments | A small proportion of PG data is centralised, most is decentralised. | A significant proportion of the PG data is centralised and the amount is increasing. | Only a small amount of the PG data is not centralised (eg funding data) | All PG course data is held on 1 central source |
10 | Process capability | “Incomplete”: Process not carried out or only partially carried out. | “Performed”: Process is carried out but improvements are not sustained. | “Managed”: Basic infrastructure exists to support the process; it is carried out by skilled people, with resources according to a policy; it is monitored, controlled and reviewed. | “Defined”: A managed process that is tailored from the organisation’s standard set of processes, descriptions and procedures. The process is documented, managed pro-actively and with a high level of understanding. | “Quantitatively Managed”: A defined process controlled using statistical or other quantitative techniques throughout its life. |
11 | Technical capability | None | There is some knowledge and understanding of the relevant technologies, but skills are not available in the organisation. | There is knowledge and understanding of the relevant technologies and some skills are available, but not all areas are covered, or there is lack of experience. | Service oriented approach and interoperability technical skills are available, but considerable effort is required to access them. | The organisation has to hand all the technical skills required and can deploy them easily. |
12 | Organisational context | The organisation is hostile to this development. | The organisation has significant management commitments that might make this development difficult at this time (eg re-structuring, mergers) | The organisation is open to developments in this area, and management structures suggest that commitment can be obtained. | The organisation is open to XCRI development at senior management levels and there is likely to be agreement from relevant technical and non-technical staff. | The organisation is committed to XCRI implementation at all levels. |
13 | Resources Minimal | Minimal | Some available, but it is known whether it is sufficient. | Adequate if nothing goes wrong. | Sufficient to implement successfully. | Plenty |
TABLE 2: XCRI READINESS ATTRIBUTE SCALES
For each attribute the most desirable value for XCRI readiness would be the rightmost value. The first three attributes are generally fixed and could not be adjusted to aid XCRI implementation; other attributes are under the control of the organisation, though not necessarily under the control of the course advertising information managers.
Definitions of the attributes and help text are provided in the template attached to this report.
In addition a diagrammatic representation of the set of attributes enabled the project to make comparisons between factors and between organisations readily. A template was developed in Excel for this purpose.
When reading the XCRI characteristics diagrams, the larger the area of blue, the greater the readiness of the institution for XCRI implementation.
Characterisation of HEI use of course advertising information
Minimum characteristics for XCRI readiness
As an aid to understanding and for comparison purposes, a diagram showing suggested minimum required characteristics for XCRI implementation was drawn up. This diagram shows that the Number of Courses, Complexity of Provision and Number of Sources are given (value=0); Quality of Authoring and Data Structures are at medium to high levels, so that transformation and mapping to XCRI can be carried out readily; Update Frequency and Audit Trail should exist in some form, so that data can be maintained; it is preferable to have some centralisation of data, so that the internal feeds to the institution’s aggregator are not too complex; processes for supply of course advertising information to the outside world should exist and should be capable of change; there should be some knowledge and understanding of relevant technologies, and the organisation should be prepared to initiate the XCRI implementation and provide some resource for it.
There are elements that can be increased to compensate for deficiencies elsewhere. For example, lack of technical capability could be rectified by allocation of resources to buy in consultants or to train existing staff; if data collection is heavily centralised already, it might be relatively easy to design and deliver new processes to produce XCRI outputs; and in many cases fewer, less complicated course provision will reduce the importance of the existing quality of the data and its structures.