Steph Buongiorno and Corey Clark. “Leveraging Gaming to Enhance Knowledge Graphs for Explainable Generative AI Applications.” Proceedings of IEEE Conference on Games, 2024, Milan, Italy.
See the pre-print: https://arxiv.org/abs/2404.19729
LLM’s application within domains where accuracy is critical–such as human trafficking data analysis–is limited due to a lack of explainability in results. A LLM’s mechanism for generating responses is opaque and may contain unwanted biases or contribute to “hallucinations” (generated text that is semantically possible but factually incorrect). High-stakes domain problems demand verifiable methods, making reliance on LLMs without transparency problematic. For instance, claims that someone broke a law could be harmful if they are based on hallucinations or lack validation. Providing LLMs with external, structured representations of facts in the form of knowledge graphs (KGs) has proven useful for addressing this limitation.
While using KGs to augment LLM responses has proven beneficial, obtaining KGs that represent the desired domain information poses challenges. KGs can be parsed from documents, but capturing the many entity relationship from within text may not be feasible with automated approaches alone. For instance, a KG parsed from text may include explicit relations between entities (e.g. semantic relations as directly expressed in words), but may not include implicit relations (e.g. relations that may be clear to a human, but not to the computer). In other cases it may be desirable to modify the explicit relationships of a KG, for instance, if the connections between entities are incorrect.
For these reasons, this research introduces the GAME-KG framework (standing for ``Gaming for Augmenting Metadata and Enhancing Knowledge Graphs”). GAME-KG is a federated approach to modifying KGs that facilitates the collection of explicit and implicit knowledge. It leverages Human Computation Gaming (HCG)–a method of collecting feedback from crowds through video games–to modify and validate KGs. GAME-KG guides the process of parsing a KG to presenting the data to the player and collecting feedback.
