Abstract:
General large language models (LLMs) have suffered from insufficient domain knowledge and professional expertise in specialized swine disease question answering (QA) tasks. It is lacking on the effective answer evaluation and optimization to reduce factual hallucinations, severely undermining the logical consistency and diagnostic accuracy of LLMs in smart agriculture. In this study, an intelligent QA framework for swine disease was proposed to integrate with knowledge routing and agent reflection mechanisms, termed Reflective-QA. The dataset was constructed for swine disease question answering. (1) A high-quality corpus of swine disease was constructed with 10 million characters covering 163 swine diseases, according to professional veterinary books, authoritative monographs, and clinical case records. Five sub-corpora were divided into infectious, parasitic, toxic, nutritional-metabolic, and common diseases, according to pathogenic characteristics. Background knowledge of the QA model was also strengthened to enhance the professionalism of answers. (2) Knowledge routing with multi-agent voting was designed to map user queries to targeted disease sub-corpora. Noise interference and retrieval explosion were effectively reduced in the process of QA. The retrieval space was realized for the purity of vectorized retrieval information. (3) A multi-agent QA mechanism was designed to collaborate with different interaction partners, including a preliminary diagnostician, veterinary expert, and swine specialist. Structured and professionally candidate answers were generated after collaboration. Furthermore, an agent reflection mechanism was established to achieve closed-loop self-correction of answer quality. Specifically, three evaluation indicators were adopted to assess answer quality, including knowledge coverage, intent understanding deviation, and role collaboration efficiency. Two optimization strategies, namely semantic rewriting and task decomposition, were deployed to refine the generated answers. Experimental results demonstrated that the framework outperformed baseline models in terms of bilingual evaluation understudy (4-gram) (BLEU-4), recall-oriented understudy for evaluation-longest common subsequence (ROUGE-L), and BERT-based score recall (
R), with the scores of 51.93%, 30.50%, and
0.7409, respectively. BLEU-4, ROUGE-L, and
R scores increased significantly by 8.43 percentage points, 1.19 percentage points, and 0.0359, respectively, compared with the baseline model, i.e., DeepSeek-R1. The improved model outperformed the currently popular general LLMs in terms of question answering performance in the field of swine diseases. In addition, the multiple-choice QA task revealed that substantial accuracy was improved for easy, medium, and hard questions, with the accuracy values of 90%, 80%, and 70%, respectively. An ablation experiment verified that the knowledge routing, multi-agent collaborative strategy, and agent reflection mechanism contributed to higher question answering accuracy. Specifically, the average accuracy decreased from 80% to 67% in multiple-choice question answering after the knowledge routing module was removed. In summary, the ambiguity and hallucination were resolved in knowledge retrieval of swine disease. This finding can provide a modular and reliable technical scheme for agricultural QA and professional LLM agent deployment. The promising generalization potential can be expected for crop protection and aquaculture in modern agriculture.