BM 10: Each course includes a clear statement of learning outcomes in respect of both knowledge and skills. There is reasoned coherence between learning outcomes, the strategy for use of e-learning, the scope of the learning materials and the assessment methods used.
Learning outcomes express what students should have achieved when they have completed the course. These outcomes will be specified in terms of knowledge, skills, vocational/professional competencies, personal development, etc. and will usually be a combination of these.
Each course should have a specification which sets out the relationship between learning outcomes and their assessment. The course design team should ensure that the assessment strategy reinforces the teaching strategy.
Decisions about the use of e-learning and e-assessment should be made on the basis of providing the most effective means of achieving and assessing the learning outcomes. There should be a clear rationale for the use of e-learning and the level of support provided.
What does this mean?
In the VTT-Box method this is fulfilled with the definition and use of the self-evaluation mandala. Additional, it is expected that teachers care for a strict planning of
- The used material
- The implemented assessment (this should alwais be a mixture of formative and summative assessment)
Recommendation: The self-evaluation mandala helps the teacher to define the competences taught in the course as well as the students to understand and to know what they will/have to learn.
BM 11: Learning outcomes determine the means used to deliver course content. In a blended-learning context there exists an explicit rationale for the use of each component in the blend.
Course designers should adopt a logical approach to their use of e-learning. Fitness for purpose should drive decisions on the selection of educational components. The blending should be such that methods and media are well chosen for the purposes they need to fulfil.
Course designers must resolve the tension between:
- the flexibility of access to resources offered by the anywhere, anytime availability of e-learning
- the flexibility of educational interaction offered by face-to-face contact between students and teachers.
If it is not possible to deliver learning outcomes solely through e-learning then some face-to-face techniques may be essential.
What does this mean?
Means for the course must be selected sereously and based on pedagogical considerations. The use of multimedia-based content or interactive content is recommended, but should not end in itself.
BM 12: Course design, development and evaluation involve individuals or teams with expertise in both academic and technical aspects.
The course design process should be carried out by staff who are equipped to address both academic and technical aspects of e-learning. Those involved in the course design process require experience of e-learning and its capabilities. In a rapidly evolving field much of this experience is gained through project participation rather than formal qualifications.
The development of high quality e-learning content is dependent on close collaboration and good communication between academic course designers and those responsible for the production of the constituent media components. Engagement between academics and media professionals in the technical design contributes significantly to the effectiveness of course materials.
A key issue that affects working relationships is whether technical design inputs should be integrated with the academic and pedagogic design process or applied to the outputs from this process. Institutions should consider the most appropriate methodologies to use in their specific context.
What does this mean?
In school situation teachers often have to develop a course on their own.
It is recommended that teachers of the same subject build a team and work together. This might also bring a split of tasks where some teachers develop the content and others well-experienced care for the transfer to the learning platform and create necessary multimedia-based content.
BM 13: OER and other third-party material is selected with regard to learning outcome, tailored if necessary for fit to the learning context, and integrated with other learning materials. These materials are subject to the same review processes as other course materials.
Open Educational Resources (OER) are digital materials offered freely and openly for use and reuse by educators and students. OER can be found through the large institutional and collaborative repositories that now exist.
A course designer could develop a course by picking existing OER components, or other third-party resources (and perhaps customising them), rather than developing new material from scratch.
The resulting e-learning material should be judged under the same quality criteria as newly-authored material or bought-in material.
An important benefit of OER is that the licence to freely change material makes it possible to update and improve it, allowing high quality e-learning components to evolve as users improve content and offer it back to the OER community.
Rights must be carefully tracked to ensure that the appropriate level of access is preserved and that authors are credited where appropriate. The intellectual property rights associated with OER (often one of the Creative Commons licences) usually allow material to be used without cost for non-commercial purposes and allow material to be freely reversioned and updated.
What does this mean?
In most subjects relevant and well-fitting content is available (for example as OERs). These materials might be used for the course, but a serious check must be done to ensure that the material is eligible for the course. Here the course developer should not make any compromise. If the material fits it can be taken otherwise it must be developed new.
Hint: Never use OER or other third party material in a foreign language (except the use of this foreign language is part of the course and fits to the course concept).
BM 14: E-learning materials have sufficient interactivity (student-to-content or student-to-student) to encourage active engagement and enable students to test their knowledge, understanding and skills.
E-learning opens up many possibilities for interactivity in learning materials and activities. Interactivity can relate to an individual student using online resources such as simulations, animations and quizzes. It can also relate to students interacting with each other via online communication tools such as forums, wikis or web conferencing.
Individual students will benefit from the active approach to learning and the feedback provided by simulations and interactive formative assessments. These techniques can encourage deeper learning.
Online communications and collaboration tools have great potential in providing support and academic community building for distributed student populations. Their use can be a make or break factor for some students, and can make the learning experience far more engaging and lively for everyone.
What does this mean?
The use of interactive and multimedia-based content is a recommendation from the pedagogical framework. This should lead to active learning which cares for longer-lasting learning results.
Hint: Interactive and multimedia-based content might also be used for assessment.
BM 15: Independent learning materials provide learners with regular feedback through self-assessment activities or tests.
Independent learning materials may be designed to serve the needs of several courses or programmes; such packages should therefore be self-contained, have clear learning objectives and measurable outcomes.
When delivered by e-learning the materials should be designed to maximise the use of interactive techniques to provide opportunity for student self-assessment of progress towards learning outcomes.
E-learning offers opportunities for embedded interactive formative assessment with automated feedback. Development of these assessments requires significant academic input and collaboration with experts in the facilities available through the institution’s VLE systems. The benefits to students through rapid feedback are considerable.
What does this mean?
Self-assessments should be used intensively. This is valid for independent learning material as well as for other material in the course.
BM 16: Courses conform to explicit guidelines concerning layout and presentation and are as consistent as possible across a programme.
This benchmark tests aspects of good practice in materials design. A poor score in this area may indicate that the institution should make a greater investment in the contribution of media professionals to course design teams.
It is recommended that institutions implement a framework of technical, accessibility and presentational standards that apply to e-learning materials. This framework should include the following factors:
- Interfaces should conform to usability and accessibility standards.
- materials should be accessible to users with special requirements, for example students with a visual impairment or limited manual dexterity.
- Learning materials should have good graphic design standards.
- Materials should be neutral as to sex, ethnicity, age and so on.
- Software should be up-to-date and platform neutral.
- course developers should take into account download times.
- Learning materials should be accessible and usable via a variety of devices including mobile devices.
- Style sheets and schemas should be used to provide consistency of presentation.
- Course developers should be provided with suitable authoring tools.
What does this mean?
Students need a strict and simple outlay of the course. No student wants to undertake a research to find the next assignment of content offered from another platform. The consistency of the course is crucial and should be checked regularly, especially after enhancements or adds in an existing course.
Hint: The use of templates may be helpful for the course development.
BM 17: Courses provide both formative and summative assessment. Assessment is explicit, fair, valid and reliable. Appropriate measures are in place to prevent impersonation and/or plagiarism, especially where assessments are conducted online.
Course design should provide opportunities for formative and summative assessment.
Continuous assessment may fulfill both formative and summative functions. Feedback from tutors/mentors on assessment tasks provides the learner with reinforcement or direction for remedial learning. Teachers should be required to provide timely feedback aimed at improvement. Marking criteria need to be uniformly understood and consistently applied.
Peer and self-review can also be used for formative assessment. Clear marking criteria are needed for this to be a valuable exercise.
The development team should aim to exploit the interactivity inherent in e-learning, particularly through formative assessment activity, to confirm and reinforce student learning. Formative assessment may be built into the design of structured teaching materials.
The use of e-learning raises issues of verification of student identity, and measures should be taken to prevent impersonation and plagiarism. These measures may include: checking identities at approved examination centres; using software to detect plagiarism and collusion; cross-referencing and correlation between performance on written examinations and on continuous assessment.
What does this mean?
Assessment is a typical issue to evaluate learning results in schools. A well-defined concept for the assessment using a variety of different but well-performed and useful assessments should be developed from the beginning.
Hints: Assessments can be used for the learning as well (for example as self-evaluations leading to case studies). Besides that, not everything must be assessed. Some freedom in learning with well-done moderation and peer learning might be more successful than annoying permanently done assessments.
BM 18: Course materials, including the intended learning outcomes, are regularly reviewed, up-dated and improved using feedback from stakeholders as appropriate.
The institution should have procedures to ensure effective review and updating processes. Independent evaluation of course design and course materials should be carried out to ensure comparability with national or professional standards.
In the case of e-learning courses, the evaluation process should address subject content, modes of delivery and levels of interactivity. For example:
- external assessors should be engaged to review course design and provide developmental feedback
- the monitoring and evaluation process should provide documented feedback for improvement and redevelopment
- whilst in presentation, data on patterns of student use may be gathered and analysed
- evaluation information should be sought via student surveys.
In an e-learning situation there is the potential for generation of very extensive data on student performance, etc. for quality improvement. The course design team should devise a strategy for exploiting this.
What does this mean?
It is good praxis to keep learning and training material up to date. As an example, the replacing of outdated images by new ones is a must, as well as replacing names (if necessary) or other issues. Regularly done feedback from students helps to identify material that should be replaced or renewed.