Quantitative and Qualitative Evaluation of a Jupyter Cell Recommender System

This work explores various developer-centric and user-centric quality attributes of recommender systems (RecSys), focusing on their application to evaluate the similar cell recommendation in JupyterRecSys – a RecSys and Jupyter Lab extension proposed by [Xu23]. A systematic methodology is used to assess the developer-centric quality attributes accuracy, diversity, coverage, robustness, confidence, adaptivity, privacy and security as well as recommendation time and scalability, alongside with the user-centric quality attributes utility, novelty, serendipity, trust and explainability as well as risk. For the quality attributes deemed suitable for the evaluation of JupyterRecSys, namely accuracy, diversity, confidence, recommendation time and scalability, utility, novelty, serendipity as well as trust and explainability, quantitative and qualitative evaluation methods are adapted to ensure their applicability within the context of JupyterRecSys. The evaluation methods of selected attributes are implemented in a created evaluation system for JupyterRecSys, resulting in its capability of creating evaluation datasets and assessing accuracy and confidence. The evaluation results indicate a remarkable accuracy for JupyterRecSys’ similar cell recommendation, highlighting its potential usefulness in providing cell recommendations in the Jupyter environment.

Project information

Status:

Finished

Thesis for degree:

Bachelor

Student:

Dennis Mertens

Supervisor:
Part of research project:

SE4ML - Processes, People and Tools

Id:

2024-008