Abstract
Post-harvest diseases of apple are one of the major issues in the agricultural sector of apple production, causing severe economical losses to producers. Thus, in this dissertation, we illustrate the development of extit{DSSApple}, an interactive web-based decision support system, that helps users to diagnose post-harvest diseases of apples based on the observed macroscopic symptoms.
Firstly, we present a picture-based and conversational interface for extit{DSSApple}, where sampled images depicting symptoms of apples with known diseases are shown to users to elicit their feedback on perceived similarities, in order to determine the most likely diagnosis of a diseased target apple. We provide, besides the description of the industrial application scenario and the design choices, multiple contributions circled around three rounds of user studies: (i) an usability and effectiveness assessment of the approach, where three user interface configurations are put to a test and the effectiveness of different types of user feedback mechanisms is assessed; (ii) contextual multi-armed bandit approaches for dynamic selection of displayed images with symptoms of diseased apples, that clearly outperform random and greedy sampling baseline strategies; (iii) a comparison of two different strategies for determining the context representation of a contextual multi-armed bandit approach, namely based on image processing and users' interactions from a gamified large-scale user study.
We further enhance the proposed picture-based system by addressing the problem of sequentially optimizing for the best diagnosis, exploiting past interactions with the system and their contextual information (i.e., the evidence provided by the users), while exploring the set of candidate diseases. We frame such a problem as a extit{Contextual Multi-Armed Bandit (CMAB)}. This allows to interactively update the decision model considering the success of each past interaction with respect to the context provided in each round. However, contextual information is very often partial and inadequate to handle a complex decision making problem. On the other hand, human-made decisions implicitly include unobserved factors (referred to as extit{unobserved confounders}) that significantly influence their choices. Thus, we take advantage of the information embedded in the observed human decisions to marginalize confounders and improve the capability of the CMAB model to identify the correct diagnosis. Specifically, we propose a extit{Counterfactual Thompson Sampling (CF-TS)}, a CMAB model based on the causal concept of counterfactual. The proposed model is validated with offline experiments simulated on data collected through a large user study on the extit{DSSApple} application. The results prove that CF-TS is able to significantly outperform both traditional CMAB algorithms and observed user decisions in the real-world task of predicting the correct apple disease.
We follow-up our research by investigating the application of a knowledge-based expert system responsible for the diagnosis of post-harvest diseases of apple. Specifically, we describe the implemented expert system, dubbed extit{BN-DSSApple}, which collects user feedback (i.e., the evidence) on the diseased target apple through a series of adaptive multiple-choice questions related to the observed symptoms' characteristics. The system is based on the extit{Bayesian Network (BN)} framework, thus, we detail the process of domain knowledge elicitation from a domain expert for developing the Bayesian reasoning system (i.e., the probability parameters connecting symptoms to diseases). We formalize the diagnostic mechanism given the evidence provided by the user, as well as an explanation algorithm based on the maximum extit{normalized likelihood} of the evidence over the suggested diagnosis. Finally, we face the extit{transferability} problem of such an expert model. Hence, we propose a extit{likelihood evidence} approach, which leverages the estimated consensus of users' and expert's interactions, to effectively transfer the performance of the model to different cohorts of users.
We evaluate extit{BN-DSSApple} with three different types of user studies involving diseased apples, whose target diseases have been determined by microbiological analysis. The experiments demonstrate the performance differences of the knowledge-based diagnostic system due to different users interacting with the system under different conditions, and the capability of the likelihood-based method to adapt to the user's diagnosis in different environments, maintaining good performances.
Finally, we present the extit{DSSApple} final version, which is designed as a two-stream hybrid diagnostic tool, that can be easily and effectively used by both expert and non-expert users. The system includes both an image-based and an expert-based interaction paradigm. The extit{image-based} stream allows the user to interact simply by selecting pictures, representing the variety of symptoms of different diseases at different stages of infection and on different cultivars. Instead, the extit{expert-based} stream of the system incrementally collects user feedback about the target disease, by asking adaptive multiple-choice questions related to the macroscopic characteristics of the observed symptoms on the target infected apple. We describe a straightforward hybridization technique for computing the diagnosis given the feedback collected in both streams. We thoroughly test the proposed hybrid approach by means of two real user studies, involving simulated (by photos) and real infected apples, with the ground-truth disease determined in the laboratory. Thus, we prove that the hybrid version of extit{DSSApple} is able to outperform both the single streams and the self-reported user intuition in terms of diagnosis accuracy.