When AI Lies With Confidence: Diagnosing LLM Truthfulness Through Neighborhood Consistency 📊

The paradox of large language models is that they've never sounded more certain—or been more wrong. We've all seen it: a chatbot delivers a perfectly grammatical, utterly confident explanation of something that simply isn't true. The model doesn't know it's lying. It can't. And that's precisely the problem.

As LLMs become embedded in everything from customer service to medical advice, the question of truthfulness has shifted from academic curiosity to existential necessity. But how do you measure something as slippery as "truth" in a statistical text generator? The answer, according to a growing body of research, lies not in what the model says, but in how consistently it says it.

The paper "Illusions of Confidence? Diagnosing LLM Truthfulness via Neighborhood Consistency" proposes a deceptively simple framework: if a model gives the same answer to semantically equivalent questions, it's likely telling the truth. If it flip-flops, you've caught it hallucinating. This tutorial walks through implementing that diagnostic from scratch, giving you a practical tool to peer inside the black box of LLM reliability.

The Architecture of Doubt: Why Consistency Beats Confidence

Before diving into code, it's worth understanding why neighborhood consistency works as a truthfulness metric. Traditional approaches to evaluating LLM truthfulness rely on confidence scores—the model's own assessment of how likely its answer is. But here's the dirty secret: confidence scores are themselves generated by the model. You're asking the liar to grade its own homework.

Neighborhood consistency sidesteps this circular logic entirely. The core insight is elegant: a truthful model should produce stable outputs across minor perturbations of the input. If you ask "What is the capital of France?" and then "Name the capital city of France," a truthful model returns "Paris" both times. A hallucinating model might say "Paris" in one case and "Lyon" in another, because its internal representation of the fact is unstable.

This approach draws on techniques from vector databases and semantic similarity analysis, where the "neighborhood" of an input is defined by semantically equivalent but syntactically varied formulations. The consistency across that neighborhood becomes a proxy for truthfulness—one that doesn't require ground truth labels or human annotation.

The implementation we'll build uses GPT-2 as a test case, but the methodology generalizes to any causal language model. We're not evaluating GPT-2's truthfulness per se (spoiler: it's not great), but rather demonstrating a diagnostic framework you can apply to any model in your pipeline.

Setting the Stage: Environment and Dependencies

The technical requirements are refreshingly modest. You'll need Python 3.10 or later, PyTorch 2.0+, and the Hugging Face ecosystem. The full dependency list is precise:

• torch >= 2.0
• transformers >= 4.25
• datasets >= 2.6
• numpy >= 1.23

Start by creating a clean environment. Virtual environments aren't optional here—dependency conflicts between PyTorch versions can silently corrupt your results, and nothing undermines a truthfulness evaluation like a broken tensor operation.

python -m venv llm_evaluation_env
source llm_evaluation_env/bin/activate
pip install torch>=2.0 transformers==4.25 datasets==2.6 numpy>=1.23

Next, scaffold your project:

mkdir llm_diagnosis_project
cd llm_diagnosis_project
touch main.py requirements.txt README.md

The requirements.txt file should pin exact versions for reproducibility. In production evaluations, you'll want to lock these down—a minor version bump in transformers can change tokenization behavior and invalidate your consistency metrics.

Building the Diagnostic Engine: Core Implementation

The heart of this system is a pipeline that loads a model, generates responses across a neighborhood of inputs, and computes consistency metrics. We'll use GPT-2 as our test subject, but the architecture supports any model from the Hugging Face hub.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from datasets import load_dataset

model_name = "gpt2"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Load a reference dataset for generating test inputs
dataset = load_dataset('wikipedia', '20200501.en')

def tokenize_function(examples):
    return tokenizer(examples['text'], padding='max_length', truncation=True)

tokenized_datasets = dataset.map(tokenize_function, batched=True)

The choice of Wikipedia as a reference dataset is deliberate. Wikipedia text provides a diverse, factual corpus from which we can extract statements and generate neighborhood variations. In practice, you'd want to curate a test set specific to your domain—medical LLMs should be evaluated on clinical texts, legal models on case law.

Hardware configuration follows standard best practices:

device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
tokenizer.pad_token = tokenizer.eos_token

Setting the padding token to the EOS token is a subtle but critical detail. Many causal language models don't have a defined padding token by default, and failing to set one will cause silent failures during batch processing. These silent failures are exactly the kind of bug that can corrupt an evaluation run without throwing an error—the model will still generate text, but the consistency metrics will be meaningless.

Generating Responses and Measuring Consistency

The evaluation function takes an input text and generates a response, then we'll extend this to generate responses across neighborhood variations and compare them.

def generate_response(text):
    inputs = tokenizer.encode_plus(text, return_tensors='pt').to(device)
    outputs = model.generate(
        inputs['input_ids'], 
        max_length=100,
        do_sample=False,  # Deterministic generation for consistency
        temperature=1.0
    )
    response_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response_text

Note the do_sample=False parameter. This is crucial for consistency evaluation—we want deterministic outputs. If the model uses sampling, the same input can produce different outputs, which would inflate our inconsistency metric and make the model appear less truthful than it actually is. We're measuring the model's knowledge, not its creativity.

To implement neighborhood consistency, we need to generate semantically equivalent variations of each test input. This can be done manually for small tests or programmatically using paraphrasing models for larger evaluations. A simple approach:

def generate_neighborhood(text):
    """Generate semantically equivalent variations of input text."""
    variations = [
        text,
        text.replace("What is", "Can you tell me"),
        text.replace("What is", "Do you know"),
        text + "?",
        text.rstrip('?') + "?"
    ]
    return variations

def evaluate_consistency(text):
    neighborhood = generate_neighborhood(text)
    responses = [generate_response(var) for var in neighborhood]
    
    # Simple consistency metric: ratio of identical responses
    unique_responses = len(set(responses))
    consistency = 1.0 - (unique_responses - 1) / len(responses)
    return consistency, responses

This is a simplified version—production systems would use more sophisticated semantic similarity metrics, possibly leveraging open-source LLMs as judges. But the principle holds: high consistency correlates with truthfulness, and inconsistency is a red flag.

Advanced Diagnostics: Beyond Binary Truthfulness

The basic consistency metric is useful, but real-world evaluations require nuance. A model might be consistently wrong—returning the same incorrect answer across all variations. This is still a failure mode, but it's different from the inconsistency that signals hallucination.

Advanced implementations should track two dimensions:

1. Internal consistency: Does the model give the same answer to semantically equivalent questions?
2. External consistency: Does the model's answer match known ground truth?

For external consistency, you'll need a curated test set with verified answers. The AI tutorials community has developed several benchmark datasets specifically for this purpose, including TruthfulQA and HaluEval.

Performance optimization is another consideration. Evaluating across neighborhoods multiplies your inference cost by the neighborhood size. For large-scale evaluations, consider:

• Batch processing all neighborhood variations together
• Using gradient checkpointing for memory efficiency
• Implementing parallel processing across multiple GPUs

from concurrent.futures import ThreadPoolExecutor

def batch_evaluate(texts, batch_size=8):
    with ThreadPoolExecutor(max_workers=4) as executor:
        results = list(executor.map(evaluate_consistency, texts))
    return results

Interpreting Results: What the Numbers Actually Mean

Running this evaluation on GPT-2 reveals something interesting: the model shows moderate consistency (around 60-70%) on simple factual questions, but that number drops precipitously for complex or ambiguous queries. This isn't surprising—GPT-2 was never designed for factual accuracy—but it validates the methodology.

More importantly, the consistency metric correlates with specific failure modes. Low consistency often indicates the model is "guess-and-checking": generating plausible-sounding text without a stable internal representation. High consistency with wrong answers suggests the model has learned a false pattern from its training data, which is harder to fix but easier to detect.

For production deployments, establish a consistency threshold based on your use case. A medical diagnosis assistant might require 95%+ consistency, while a creative writing tool could tolerate lower scores. The key insight is that you now have a quantitative tool for making that decision, rather than relying on vibes and anecdotal evidence.

The Road Ahead: From Diagnosis to Treatment

Neighborhood consistency is a diagnostic tool, not a cure. But diagnosis is the first step toward treatment. Once you've identified which inputs trigger inconsistency, you can:

• Fine-tune the model on those specific failure cases
• Implement retrieval-augmented generation (RAG) to ground responses in verified sources
• Build confidence thresholds that trigger human review for low-consistency responses

The field is moving rapidly. Newer models like GPT-4 and Claude show dramatically higher consistency, but they're not immune to the fundamental problem. As LLMs become more capable, the consistency diagnostic becomes more nuanced—but no less necessary.

The uncomfortable truth is that we're deploying systems we don't fully understand into increasingly critical roles. Neighborhood consistency gives us a window into their internal reliability, a way to measure something we can't directly observe. It's not perfect, but it's a start. And in a world where AI-generated text is becoming indistinguishable from human writing, imperfect diagnostics are infinitely better than blind trust.

---