通讯作者: 高东平, gaodp_gaodp@126.com

DOI：10.12201/bmr.202505.00042

Construction of Multimodal Nutrition Knowledge Graph

• 摘要：饮食是人们生活中的关键一环。近年来,随着生活水平的提高,人们越来越注重饮食的健康与个性化。为了精准、有效、直观地为不同人群提供营养和饮食建议,构建多模态营养知识图谱是十分必要的。本文结合营养学书籍、文献和网站中的营养数据,构建了包含食物、营养、疾病等实体的多模态营养知识图谱。借鉴OneRel模型,完成中文实体关系联合抽取。通过利用感知哈希(pHash)算法对获取的食物图像数据进行过滤,使用RoBERTa-ResNet模型分别学习文本和图像数据特征,并文本、图像特征向量进行拼接得到融合向量,通过加入全连接层学习模态间的表层特征,辅助构建多模态知识图谱。最后,利用Neo4j图数据库对多模态营养知识图谱进行存储和可视化展示。本文提出的跨模态领域知识图谱构建方法构建的营养多模态知识图谱不仅能系统化地整合营养领域多模态知识,实现良好的可视化查询,也是智能问答、营养推荐系统等下游任务的底层支撑。

  关键词： 多模态知识图谱; 知识表示; 健康饮食

   

  Abstract: Diet is a crucial aspect of peoples lives. In recent years, with the improvement of living standards, people have increasingly focused on the health and personalization of their diets. To provide precise, effective, and intuitive nutritional and dietary recommendations for different populations, it is essential to construct a multimodal nutrition knowledge graph. This paper constructs a multimodal nutrition knowledge graph that includes entities such as food, nutrition, and diseases by integrating nutritional data from nutrition books, literature, and websites. By referencing the OneRel model, joint extraction of Chinese entity relationships is completed. The perceptual hash (pHash) algorithm is used to filter the acquired food image data, and the RoBERTa-ResNet model is employed to learn the features of text and image data separately. The text and image feature vectors are concatenated to form a fused vector, and a fully connected layer is added to learn the superficial features between modalities, which aids in the construction of the multimodal knowledge graph. Finally, the Neo4j graph database is utilized to store and visually display the multimodal nutrition knowledge graph. The multimodal nutrition knowledge graph constructed using the cross-modal knowledge graph construction method proposed in this paper not only systematically integrates multimodal knowledge in the field of nutrition and enables good visual query capabilities but also serves as the underlying support for downstream tasks such as intelligent question answering and nutrition recommendation systems.

  Key words: Multimodal knowledge graph; Knowledge representation; Healthy diet

  提交时间：2025-05-27

  版权声明：作者本人独立拥有该论文的版权，预印本系统仅拥有论文的永久保存权利。任何人未经允许不得重复使用。
• html

• 图表

• 吴萌, 杨林, 沈柳, 张素菡, 王敏, 孙振凤, 徐晓巍, 刘娜娜, 王亚新, 侯丽, 李姣, 马良坤. 面向继续医学教育的多模态围产保健知识图谱构建研究. 2024. doi: 10.12201/bmr.202402.00008

  王华琼, 俞定国, 钱归平. 基于医学社交媒体数据的多模态知识图谱构建. 2022. doi: 10.12201/bmr.202209.00005

  刘燕, 张潇潇, 侯丽. 面向知识服务系统的学术知识图谱构建与应用研究. 2024. doi: 10.12201/bmr.202402.00015

  方攀, 曹宇汀, 丁子啸, 张顺, 李兆融, 曾震宇, 朱睿. 老年主动健康知识图谱构建和应用探索. 2023. doi: 10.12201/bmr.202303.00038

  胡红娟, 周阳, 匡泽民, 谭琳. 医学知识图谱应用研究进展. 2021. doi: 10.12201/bmr.202107.00012

  梁静, 文奕. 知识图谱在医学辅助诊断中的应用研究. 2022. doi: 10.12201/bmr.202109.00021

  付涛涛, 陈艳梅, 李庆娜, 邵义明, 苏国彬, 弓孟春. 基于《中国药典》的中药知识图谱的构建与应用. 2024. doi: 10.12201/bmr.202407.00039

  吴欢, 何昆仑. 基于循证医学和电子病历数据的通用医学知识图谱构建. 2024. doi: 10.12201/bmr.202409.00027

  陈婕卿, 竹志超, 张锋, 曾可, 姜会珍, 程振宁. 面向知识图谱构建的中文电子病历命名实体识别方法研究. 2023. doi: 10.12201/bmr.202312.00011

• 序号	提交日期	编号	操作
  2	2025-05-23	bmr.202505.00042V2	下载
  1	2025-05-23	bmr.202505.00042V1	下载

• 公开评论  匿名评论  仅发给作者 提交评论