Quality assurance in higher education (QAHE)
Since the inception of the European Higher Education Area (EHEA), which stems from the Bologna Declaration (1999), fostering the comparability and recognition of qualifications has made higher education evaluation a priority in education policy. From an incipient and contested initial system of quality assurance set up in the late 20th century (Law No 38/94, 21 November), higher education evaluation in Portugal has expanded rapidly, today boasting high-quality supervisory and consultancy services, no longer focussing merely on teaching-learning processes and results achieved, but including an analysis of course organisation and operation.
In 1998, by Decree-Law No 205/98, 11 July, the National Board for Higher Education Assessment (Conselho Nacional de Avaliação do Ensino Superior – CNAVES) was created, aiming to ensure the higher education system’s “harmony, cohesion and credibility”, using meta-evaluation and, if necessary, foreign experts. In 2000, the second evaluation cycle was completed, covering all higher education institutions (HEIs), however, the process was contested and considered ineffective due to its considerable dependence on higher education institutions, not producing results in terms of eradicating cases of inferior quality.
In 2005, the Portuguese government requested an assessment of the quality assurance system in Portuguese higher education from the European Association for Quality Assurance in Higher Education (ENQA), to offer recommendations for improvement and for the creation of a system that would comply with Standards and guidelines for quality assurance in the European Higher Education Area (ESG). In 2007, the new Law on Higher Education Evaluation, No 38/2007, 16 August was published and the Agency for Assessment and Accreditation of Higher Education was created via Decree-Law No 369/2007, 5 November, with the main objective of promoting, assessing and ensuring quality higher education.
Both the legal framework for higher education quality assurance (Law No 38/2007, 16 August) and its quality evaluation system are universal, compulsory and frequent in nature. They require quality policies within the HEIs themselves; quality criteria being submitted to the standards stemming from the Bologna Process; assessment of R&D activities; participation of foreign experts in the evaluation process; student intervention, etc. As such, evaluation in higher education is considered clearly supervisory in nature.
The objectives of higher education quality assurance are:
a) to improve the quality of HEIs.
b) to provide society with information on the HEI performance quality.
c) to promote an internal quality assurance culture within HEIs.
In 2012, the first regular accreditation cycle began. It consisted of a five-year evaluation and accreditation phase of all operating study cycles with preliminary accreditation. It was concluded in 2017/18 with the institutional assessment process, as part of the reconstruction of the higher education database. The aim was to provide a global vision of the system and an analysis of the improvements achieved in relation to the initial 2010 undertaking.
The second evaluation cycle started in 2017, aiming to assess and renew the accreditation of the study programmes that institutions wanted to keep running after the end of the normal period of validity granted in the first regular cycle or of the initial accreditation. The second institutional assessment began in 2023 and is expected to be completed in 2024. The second regular accreditation cycle, also under the responsibility of A3ES, ran from 2018 to 2023. This cycle concluded with a new institutional evaluation process (2023/24).
Meanwhile, the third cycle of study programme assessment began in 2023.
The Higher Education Evaluation Law includes regular international evaluation of the Agency (Article 25, a). After the experience of conducting assessment and accreditation processes, the A3ES fulfilled the requirements for an independent international review, as determined by the ESG. This evaluation was a condition for A3ES’s application to be included on the European Quality Assurance Register (EQAR). The external evaluation of A3ES was coordinated by ENQA, and the external evaluation panel visited the Agency on 15-16 January 2014. In March 2014, ENQA submitted its report and that year the result of the evaluation allowed A3ES to become a full member of ENQA and be registered in EQAR. The evaluation was repeated in 2019 (five-year cycle), again with positive results, and A3ES’s registration in EQAR was renewed until 30/06/2024. A new evaluation started at the end of 2023.
The Agency for Assessment and Accreditation of Higher Education (A3ES) is also a member of the CHEA International Quality Group, the SIACES (Sistema Iberoamericano de Aseguramiento de la Calidad de la Educación Superior) and of the IMHE programme from OECD. A3ES also took part in setting up the CPLP Forum, which brings together almost all the accreditation and regulatory agencies from the CPLP countries.
Portugal has a quality assurance agency that monitors policies, such as those that involve creating a strategy, with specific and measurable objectives, tied to social dialogue, focussed on equality, equity and social aspects in higher education.
Approaches and methods for quality assurance
The A3ES quality assurance approach is based on an “external quality assurance” model, a combination of internal quality assurance system, self-assessment and external evaluation, in which the HEI’s internal monitoring and evaluation processes are analysed and used in the external quality audit and accreditation process. It adheres to standards defined in the European standards and guidelines (ESG), transposed into national legislation that establishes educational institutions as those primarily responsible for the quality of the education provided.
Every HEI is responsible for defining the internal evaluation model. Each has a duty to adopt a quality assurance policy for their study cycles, ensuring appropriate procedures, promoting a culture of quality and quality assurance, as well as developing and implementing a continuous quality improvement strategy.
There are core regulations stemming from the law regarding internal evaluation that HEIs must comply with. Among the general requirements of higher education institutions (Law No 62/2007, 10 September, Art. 40) is ensuring teacher, researcher, and student participation regarding how the establishment is governed through the general council. Students should represent at least 15 % of general council members. In terms of the legislative structure that formalised the HEI evaluation system, student participation takes other forms, such as their presence on the advisory council, in the self-evaluation processes associated with institutions’ internal quality assurance systems, and external evaluation via external assessment teams (Comissões de Avaliação Externa – CAE).
Internal evaluation takes place annually and its results must be made available to the A3ES. In turn, the A3ES provides a set of internal evaluation recommendations for HEIs in its evaluation manual (e.g., measures to prepare the institution for internal evaluation and the self-evaluation report). The broad participation of stakeholders in the diagnosis, implementation, evaluation and review – at every stage of the quality cycle – is important and, for example, the use of SWOT exercises involving students, teachers, employees and employers in the institution’s diagnosis is valued.
As part of their self-evaluation, HEIs should:
a) define formal procedures for the approval, monitoring and periodic evaluation of their study cycles, which must include the participation of pedagogical councils and students’ opinion (through their associations); of research centres that are involved in the organisation and operation of study cycles; of external consultants working with the institution.
b) adopt procedures that ensure that teaching staff have the necessary qualifications and skills to perform their duties. These must be made available to those responsible for the external evaluation processes and assessed in the evaluation reports.
c) ensure that the teaching resources available are sufficient and appropriate for each of their study cycles.
d) collect, analyse and use relevant information to effectively manage their study cycles and other activities.
e) regularly publish updated, impartial and objective quantitative and qualitative information about the study cycles they teach and the degrees and diplomas they award, monitor graduates’ progress for a reasonable period of time, in terms of employability.
These are some of the aspects that should be analysed as part of quality assurance.
Evaluation is obligatory for higher education institutions. The external evaluation of institutions and study cycles for which A3ES is responsible has a six-year cycle. The first five years focus on study cycle evaluation, while the sixth year concentrates on institutional evaluation.
Evaluation focusses are defined in the current legislation, especially in Law No 38/2007, 16h August, amended by Law No 79/2019, 4 September and include:
1) scientific level of teaching.
2) teaching and learning methodologies.
3) student evaluation processes.
4 teaching staff qualification and appropriacy to study cycles analysed.
5) the efficiency of the internal quality assurance system.
6) the institution’s scientific, technological and artistic activity.
7) facilities and didactic/scientific equipment.
8) social support mechanisms.
External evaluation is based on the work of the external assessment teams (CAE), blending documentary analysis of self-evaluation reports – based on a set of criteria that includes widespread stakeholder participation – implementation data and other institution documents. It ends with a CAE visit to premises, conducting interviews with management, and with focus groups, including students, faculty, and other stakeholders.
The CAE are made up of between three and five independent experts who have no relationship with the higher education institution being evaluated. They are appointed by the A3ES management board and must include one student and one international expert. All CAE visits are accompanied by an A3ES procedure manager. This manager is not a commission secretary, but a professional highly trained in evaluation methodologies, who accompanies the commission. The standards of conduct for A3ES employees are set in the code of ethics, which are part of the quality manual, including standards for CAE designation and conduct.
The members of the CAE attend a training session specifically designed and delivered by A3ES. They are responsible for preparing for their duties via prior analysis and study of documents, such as the evaluation manual, the evaluation/accreditation of study cycles guide and, in particular, the self-evaluation reports corresponding to the study cycles being analysed.
The evaluation results are based on explicit and published criteria. These are consistently applied and found in the evaluation manual, which is available on the A3ES website.
Guides and manuals for procedures regarding the various components of the quality assurance system are also available, particularly on how to conduct the accreditation process to ensure some uniformity of procedures among the different CAE.
The CAE prepares the preliminary external evaluation report. At this time, HEIs are entitled to comment. The CAE publishes the final report and the A3ES board of directors decides on accreditation. In conditional cases, an assessment follow-up system may be required. The management board makes the final decision on accreditation. Where there are professional associations, their opinions are considered. The management board may make decisions that do not coincide with the CAEs’ or professional orders’ recommendations. Because the management board is responsible for ensuring the equity and balance of final decisions, it may decide in favour (be less demanding than the team) or against (be more demanding than the team) the CAE’s recommendation. The management board’s decisions may be reviewed by the appeals council on appeal by the HEIs.
According to Article 16 of Law No 37/2007, 16 August, evaluation results are public and published on the A3ES and HEI website/study cycle evaluated. Also, the contradiction (pronouncement) to the external evaluation reports, if there is one, must be published alongside the final version of the same reports. In addition to this, the applicable legislation (Article 16(2) of Law No 38/2007, 16 August) requires higher education institutions to publish both the self-evaluation and external evaluation reports.
Non-accreditation of a study cycle means that the HEI cannot accept new students but may continue to operate for two years to allow students to complete their studies. Non-accreditation of an institution leads to a proposal to Government for closure. In either case, measures are taken to ensure that students are not disadvantaged.
During the regular evaluation/accreditation cycle (2018-2023), the Agency developed a simplified system that was based on the principles of risk management and institutional responsibility for the quality of educational provision. The simplified procedures are based on a sampling system combined with institutional audits. They will be adopted for institutions with better quality indicators, in terms of teaching staff qualifications and the quality of research undertaken, as well as performance in the first assessment/accreditation cycle and duly certified internal quality assurance systems. This simplification of the system also means a revision and simplification of all guidelines.
A3ES quality policy is based on a clear definition of mission and objectives within the framework of European standards and guidelines for quality assurance and the applicable legal provisions. It is based on strategic planning of its activity, as demonstrated in its multiannual strategic plan and annual activity plans, sustained monitoring and continuous improvement mechanisms, and accountability, which assures transparency in all activities.
A3ES adopts a code of ethics that is applicable to all agents that work with the A3ES, which promotes the development of a culture and awareness of quality in Agency workers (internal and external). It establishes monitoring and continuous improvement mechanisms for its work, as well as undertaking systematic research, development studies and projects for further development of issues and mechanisms related to quality assurance and regular external evaluation of the Agency in accordance with European standards.
Quality Assurance in Higher Education Manual : https://ec.europa.eu/programmes/erasmus-plus/project-result-content/ed7c0c0b-3b39-4388-b686-f0b4ae96e147/Quality_Assurance_Manual_final_version_29.04.2018.pdf
