Please use this identifier to cite or link to this item: http://dspace.iitrpr.ac.in:8080/xmlui/handle/123456789/4566
Full metadata record
DC FieldValueLanguage
dc.contributor.authorGupta, S-
dc.contributor.authorJain, S-
dc.contributor.authorJha, S S-
dc.contributor.authorHsiung, P-A-
dc.contributor.authorWang, M-H-
dc.date.accessioned2024-05-29T14:13:12Z-
dc.date.available2024-05-29T14:13:12Z-
dc.date.issued2024-05-29-
dc.identifier.urihttp://dspace.iitrpr.ac.in:8080/xmlui/handle/123456789/4566-
dc.description.abstractAbstract: Training the machine learning (ML) models require a large amount of data, still the capacity of these models is limited. To enhance model performance, recent literature focuses on combining ML models’ predictions with that of human experts, a setting popularly known as the human-in-the-loop or human-AI teams. Human experts can complement the ML models as they are well-equipped with vast real-world experience and sometimes have access to private information that may not be accessible while training the ML model. Existing approaches for combining an expert and ML model either require end-to-end training of the combined model or require expert annotations for every task. End-to-end training further needs a custom loss function and human annotations, which is cumbersome, results in slower convergence, and may adversely impact the ML model’s accuracy. On the other hand, using expert annotations for every task is also cost-ineffective. We propose a novel technique that optimizes the cost of seeking the expert’s advice while utilizing the ML model’s predictions to improve accuracy. Our model considers two intrinsic parameters: the expert’s cost for each prediction and the misclassification cost of the combined human-AI model. Further, we present the impact of group-wise calibration on the combined model that improves the overall model’s performance. Experimental results on our combined model with group-wise calibration show a significant increase in accuracy with limited expert advice against different established ML models for the image classification task. In addition, the combined model’s accuracy is always greater than that of the ML model, irrespective of the expert’s accuracy, the expert’s cost, and the misclassification cost.en_US
dc.language.isoen_USen_US
dc.subjectK-way Classificationen_US
dc.subjectHuman-AI Teamen_US
dc.subjectModel Probabilitiesen_US
dc.subjectHuman-in-the-loop modelen_US
dc.subjectDeferred Modelen_US
dc.subjectCalibrationen_US
dc.titleTake Expert Advice Judiciously: Combining Groupwise Calibrated Model Probabilities with Expert Predictionsen_US
dc.typeArticleen_US
Appears in Collections:Year-2023

Files in This Item:
File Description SizeFormat 
Full Text.pdf744.26 kBAdobe PDFView/Open    Request a copy


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.