How to Build a Knowledge Graph from Documents with LLMs

Table of Contents

• How to Build a Knowledge Graph from Documents with LLMs
  • Introduction & Architecture
  • Prerequisites & Setup
  • Core Implementation: Step-by-Step
    • Step 1: Load and Preprocess Documents
• Example usage
  • Step 2: Entity Recognition with LLMs
• Example usage
  • Step 3: Relationship Extraction
• Example usage

📺 Watch: Intro to Large Language Models

Video by Andrej Karpathy

---

Introduction & Architecture

In this tutorial, we will explore how to build a knowledge graph from unstructured documents using large language models (LLMs). This process involves extracting entities and relationships from text data and representing them in a structured format that can be queried and analyzed. The architecture leverag [3]es the power of LLMs for natural language understanding tasks such as named entity recognition, relation extraction, and semantic parsing.

The knowledge graph we will construct is particularly useful for applications like recommendation systems, search engines, and data analytics platforms where contextual information plays a crucial role. As of April 18, 2026, this approach has gained significant traction in the industry due to its ability to handle complex textual data effectively.

Prerequisites & Setup

To follow along with this tutorial, you will need Python installed on your system along with several libraries that facilitate working with LLMs and knowledge graphs. The following dependencies are required:

• transformers [5]: A library for state-of-the-art models in NLP.
• spacy: For advanced natural language processing tasks.
• networkx: To create, manipulate, and study the structure of complex networks.

pip install transformers spacy networkx

Ensure you have a recent version of spaCy installed with appropriate language models. You can download the English model using:

python -m spacy download en_core_web_sm

Core Implementation: Step-by-Step

Step 1: Load and Preprocess Documents

First, we need to load our documents into a format that is suitable for processing by LLMs. This involves tokenization, lemmatization, and removal of stop words.

import spacy

def preprocess_documents(documents):
    nlp = spacy.load("en_core_web_sm")
    processed_docs = []

    for doc in documents:
        # Tokenize the document
        tokens = [token.text for token in nlp(doc)]

        # Remove stop words and lemmatize
        filtered_tokens = [token.lemma_ for token in nlp(doc) if not token.is_stop]

        processed_docs.append(filtered_tokens)

    return processed_docs

# Example usage
documents = ["John Doe works at Google.", "Jane Smith is a data scientist."]
processed_documents = preprocess_documents(documents)

Step 2: Entity Recognition with LLMs

Next, we use an LLM to perform named entity recognition (NER) on the preprocessed documents. This step identifies key entities such as people, organizations, and locations.

from transformers import pipeline

def extract_entities(processed_docs):
    ner_pipeline = pipeline("ner", model="dslim/bert-base-NER")

    entities = []

    for doc in processed_docs:
        # Convert tokens back to a single string
        text = ' '.join(doc)

        # Extract named entities
        entity_list = ner_pipeline(text)

        entities.append(entity_list)

    return entities

# Example usage
entities = extract_entities(processed_documents)

Step 3: Relationship Extraction

Once we have identified the entities, we need to determine how they relate to each other. This can be achieved by training a model on labeled data or using an existing LLM that has been fine-tuned for relation extraction.

def extract_relations(entities):
    # Placeholder function - use a pre-trained model or custom pipeline here
    relations = []

    # Example: Extracting simple relationships like "works at"
    for entity_list in entities:
        for ent in entity_list:
            if ent['entity'] == 'B-PER' and ent['word'].lower() != 'john':
                continue

            if ent['entity'] == 'I-ORG':
                relations.append((ent['word'], 'WORKS_AT', previous_entity))

            # Store the last person entity for relation extraction
            if ent['entity'] in ['B-PER', 'I-PER']:
                previous_entity = ent['word']

    return relations

# Example usage
relations = extract_relations(entities)

Step 4: Constructing the Knowledge Graph

Finally, we construct a knowledge graph from the extracted entities and relationships. This involves creating nodes for each entity and edges that represent their relationships.

import networkx as nx

def build_knowledge_graph(relations):
    G = nx.Graph()

    # Add nodes (entities)
    for relation in relations:
        G.add_node(relation[0])

        if len(relation) > 1:  # Check if there's a relationship specified
            G.add_edge(relation[0], relation[2], type=relation[1])

    return G

# Example usage
knowledge_graph = build_knowledge_graph(relations)

Configuration & Production Optimization

To scale this solution for production, consider the following optimizations:

• Batch Processing: Process documents in batches to handle large datasets efficiently.
• Asynchronous Processing: Use asynchronous programming techniques to process multiple documents concurrently.
• GPU Acceleration: Leverage GPUs for faster processing of LLM tasks.

For batch processing, you can modify the preprocess_documents and extract_entities functions to accept lists of document IDs rather than raw text. This allows you to fetch and preprocess documents in batches from a database or file system.

Advanced Tips & Edge Cases (Deep Dive)

Error Handling

Ensure robust error handling for cases where LLMs might fail due to input size limitations, model-specific errors, or unexpected data formats.

try:
    entities = extract_entities(processed_documents)
except Exception as e:
    print(f"Error processing documents: {e}")

Security Risks

Be cautious of prompt injection attacks when using LLMs. Always sanitize inputs and validate outputs to prevent malicious users from manipulating the model's behavior.

Results & Next Steps

By following this tutorial, you have built a basic system for extracting knowledge graphs from unstructured documents using LLMs. The next steps could include:

• Fine-tuning [1] Models: Improve entity and relation extraction accuracy by fine-tuning models on domain-specific datasets.
• Graph Visualization: Use tools like Graphviz or D3.js to visualize the constructed knowledge graph.
• Integration with Search Engines: Integrate the knowledge graph into a search engine for enhanced query capabilities.

This approach provides a solid foundation for building more sophisticated applications that leverage the power of LLMs and structured data.

---