Research Highlights

There are many social service projects, often funded by charity organizations, supporting various social needs, such as elderly care, parenting and special education needs. These projects need third-party evaluation to ensure their effectiveness. Traditionally, such evaluations are mainly based on questionnaires (quantitative) or interviews (qualitative) from the help recipients. Collecting such self-report (subjective) feedback has various challenges, and its accuracy has some known limitations. Given the advances of IT and AI, it is often possible to collect data from the intervention process itself and do evaluation. Such intervention-based evaluation can help evaluate the effectiveness of the intervention and often explain what aspects of the intervention worked or did not work, hence help improve the intervention. It can also be used to complement self-report-based evaluation. Furthermore, such intervention-based intervention can often be carried out using AI (and Large Language Models), making it quite efficient. Our research focuses on developing this approach of evaluation and validating the accuracy of using AI for such evaluation. Through our role in supporting several social service project evaluation projects (for example, ParentChat, ClapTech Pathway, and 656-Carer Hub), we can make use of data from these projects for our research.