In the current context produced by the COVID19 crisis, for which many learnings have been forced to make a prompt transformation to e-learning or blended learning, it is especially important to provide teachers with resources easy to use and implement, and that allow the students to go forward autonomously in their learning, minimizing the use of streaming lectures in favor of the use of specific educational materials for e-learning. It is particularly important for computer science education, since, according to studies by CEDEFOP, INE (Spanish Institute of Statistics) and SEPE (Spanish Employment Service), software development is the second most demanded profession in Europe, being the first in Spain and Portugal, and has an unemployment rate well below of the average and a much higher rate of permanent contracts. The objective of this project is to create a tool, JuezLTI, that allows the automatic assessment of exercises for which there is currently no tool (e.g. markup languages), as well as another kind of computer exercises (programming and databases). The other innovation that the tool raises is its interoperability with any online learning environment (LMS) such as Moodle, Sakai, Canvas or Blackboard (among many others) thanks to the use of the LTI standard. This, added to the pool of exercises to be developed, will allow it to be used and implemented in a large number of institutions, some of which have shown interest and explicit support for the project. The consortium that has been formed for the development of this action has agents of all the profiles necessary to successfully achieve its objectives: representatives of higher academic training, specialists in e-learning, an R&D institution with extensive experience in development of tools to support training in computer science, an e-learning software development company, a reference VET center in e-learning for Computers and Communications, and a Provincial Directorate of Education, with more than 2000 schools under its charge. Two intellectual products will be developed during the project. The first one will be a tool to support computer science teaching, JuezLTI, which will allow the automatic assessment of programming exercises, databases and markup languages, in addition to providing a pool of problems with adaptive training itineraries for the educational levels of VET and University. The IO2 is directly related to the IO1; it is a MOOC to train teachers in the use, implementation and development of the JuezLTI tool. In addition, a training activity will be carried out to share the starting knowledge of the partners in the matter, which are very diverse: e-learning methodologies, development of teaching support software, integration tools in LMS, etc. The methodology planned for the elaboration of the intellectual products will be SCRUM, which allows an agile and supervisable development, in addition to defining its own quality metrics. On the other hand, a JuezLTI pilot test will be carried out, designed as a statistical study to be executed on experimental and control groups in order to verify the improvement that the use of the tool supposes in the students. The results of this study will be published in specialized academic journals. It is expected, therefore, that as a result of implementing JuezLTI in programming teachings, the academic performance of students will improve, by proposing self-evaluating tasks and itineraries adapted to each profile. The expected impact goes beyond the scope of the institutions that make up the partnership, since a broad dissemination plan and the facilities that JuezLTI will provide for teaching computer science make it foreseeable that it will be adopted by a good number of organizations. Furthermore, the Apereo Foundation umbrella that explicitly supports this project is an indication of its sustainability.
The main objective was to integrate the benefits of self-assessment offered by virtual examiners into the day-to-day teaching of computing subjects. This meant that the feedback provided by these virtual judges should be automatically received by the institutions' LMS, thus enabling the generation of learning paths adapted to the different performance levels of the students. Furthermore, this benefit should be extended to other areas of computing. In the case of our project, we extended it to the areas of databases and markup languages. A final objective was to remove barriers to the adoption of this tool: the initial generation of exercises, for which we proposed the creation of a bank of exercises in the areas covered by the project; and training in the tool, for which we needed an open online course that could be used by any teacher in the world. Moreover, these challenges of communication between self-assessment and LMS had to be addressed without further delay, as the COVID-19 pandemic had converted the entire training offer to blended or distance learning.
As a computer science training centre with more than 30 years of experience, the project coordinating centre was aware of the existence of virtual judges of programming exercises used by teachers as an auxiliary element in their classes. These virtual judges were limited to the teaching of programming, so that the advantage of their use in teaching could not be exploited by other areas of computer science. Moreover, the use of virtual judges was usually limited to suggesting exercises for more advanced students. However, the fact that they were external tools to the institution's Learning Management System (LMS) prevented us from exploiting the potential offered by the adaptation of training programmes through the use of self-assessment tests, despite the significant improvement in student performance that they bring.