5.1 Ongoing Monitoring and Data
We would like to start this topic by a 9 minute video with power points from Mr.Gino Schiavone. This video helps you to go step by step on what data to collect and how to carry its monitoring.
Gino Schiavone is the Head of Quality Assurance at the Institute of Tourism studies (ITS Malta) and an ECVET National expert (Malta). He is qualified in Tertiary Education Management with a specialization in Quality assurance at EQF Level 7 from the University of Melbourne Australia. Gino Schiavone also developed the first accredited EQF Level 7 Internal Quality assurance course with 34 ECTS to be delivered online. His recent paper is entitled Internationalization and Quality Assurance: A European framework perspective.
Video
In this 9 minutes video is explained why is ongoing monitoring important and how HEIs leader can benefit from it.
.
Reading
Introduction: As a leader your ability to make sense of the organisational context and wider stakeholder participation is important for you to comprehend what type of ongoing monitoring is carried out and how to collect and interpret Data for any desired actions.
Please read: How should we measure Higher education? A Fundamental review of Performance Indicators Part Two. The evidence report by the Institute for Employment studies November 2013. Focus on 2 important important tables Table 2.1 on page 8 Table 2.2 (p.14) and Appednix C (p.102)
Moreover you also have case studies for the development and use of Performance Indicators in US, Canada and Australia (Pages 14 to 20 on document). A.(15 min reading)
https://dera.ioe.ac.uk/18967/2/2013_ukpireview2.pdf
Below are the Lecturers Notes about Ongoing Monitoring and Data
Benchmarking / Sharing Data
There are challenges in collecting data and sharing it within the Institution and also with other Institutions. The challenges mainly arise when data uncovers negative elements rather than positive elements. Therefore when leaders say that their Institution is open and transparent in all senses one has to see to what extent is it open and transparent. Think about it as leaders. Is there any type of secrecy within your Institution? Or is it that all departments have their ongoing monitoring, and is data shared as long as it is in line with GDPR?
The leaders can easily identify the benefits and they might have no problem in sharing data which shows benefit. Leaders must encourage a culture of open and transparent data respecting GDPR requirements. It is of utmost importance to gain agreement from the beginning between all key stakeholders of the importance of data gathering. Leaders must be able to identify what evidence is needed to show the Institutions success but also to report data which shows negative areas and any possible actions. Who will follow up these possible actions? Who will report?
So the Institutional preparation as guided by the leaders must ensure what is needed to measure on a daily, weekly, monthly and yearly basis from all departments and stakeholders. A proper plan must be available to all how this data from all departments and stakeholders will feed into the system and to show if to continue going in the same direction or if any changes of direction are required. The leader must give proper guidance on processes to be used to overcome any challenges in ongoing monitoring and data gathering. Therefore, the Institution must be equipped with clear tools and templates for data collection and its management.
What are these tools? So the leader will need to oversee that the templates are formulated in a way that can capture data as required. Another tool is the costing tool which depends on the Institution how it deals with Finance and Procurement Services. So the leader will need to carry out an exercise to calculate the benefits emanating from the cost. This will need to be captured as a whole picture for example on a spreadsheet showing benefits and their realisation. Ultimately the Institution must decide who reports about these benefits – which one of the leaders?
Some very common Baseline
Data include student questionnaire / survey results, number of applications against number of enrolments per programme, audit results and many more as fit for purpose to the Institution.
The leader must give the direction on the appropriate focus which is fit for purpose for the Institution. So the leader must question:
Key questions
- Will this data show any benefits or negative areas?
- Is this data about efficiency or financial connotations?
- How does this data show improvement?
- Is this data about satisfaction of staff, students, stakeholders?
- Will this data give enough information for enhancement?
It is of utmost importance that a desirable number of questions are well written and placed along the questionnaire to gain maximum benefit. And any former surveys are also important to check and see if the results desired actually are similar to the ones being sought after nowadays.
The leader is also responsible for disseminating data as required respecting GDPR in order to get feedback from all stakeholders. The key findings are important to show and what do they mean for improvements or motivation of staff and students. So in other words the leader will not only disseminate but he has to explain and show the benefits or areas required to act upon. A visual representation is the most effective for a leader to carry out this exercise. It is exactly like when you have a diagram with a few words of explanation underneath. PowerPoint presentations can be an alternative for good communication, depending on how these are designed and to which stakeholders they are being presented.
For example if the leader is creating an Action Plan for Improvement through a PowerPoint presentation it must also include in some slides further opportunities for data measurement or if the data captured, gave the necessary intended results.
However, the leader must make the right choice for selection of data required and its ongoing monitoring because data gathering can be a never ending story. So as part of ongoing monitoring every 6 months or every year a post-mortem discussion with all departments needs to be carried out to evaluate whether all this data is still necessary or a new type of data needs to be measured. A review of the data to be collected and its monitoring can be done every 5 years. Would you retain the same after 5 years? Can you indicate any changes the Institution has carried out through its ongoing monitoring and how has this benefitted management?
Excercise:
Choose 3 sources of data which you (would like to) collect and answer key questions (15 minutes work)
Fill in GDPR Table and post/discuss comments on two of your colleagues work or discuss it internally at your Institution.
Further reading
Moreover you also have case studies for the development and use of Performance Indicators in US, Canada and Australia.
Also you can go to Page 59 where you will find the challenges and specific criticism. If you have the time read it all as it is very informative and you can always make comparisons with your Institution. (5 min reading)
https://dera.ioe.ac.uk/18967/2/2013_ukpireview2.pdf
