What Karpathy's LLM Wiki Is Missing (And How to Fix It)

Stop Re-Deriving Knowledge on Every Query

Imagine being able to access a vast, interconnected network of knowledge without having to re-derive it every time you need it. Sounds like a dream come true, right? Well, Andrej Karpathy’s LLM Wiki pattern has indeed brought us closer to this reality, with over 5,000 stars and 3,700 forks on GitHub. However, beneath its surface lies a hidden truth: three structural gaps that can break down at scale, rendering the pattern less effective than we’d like.

Keyword Placement: What Karpathy’s LLM Wiki Is Missing

To recap, the core insight of Karpathy’s LLM Wiki pattern is to stop re-deriving knowledge on every query and compile it once into a structured wiki. This approach leverages the power of large language models (LLMs) to perform bookkeeping tasks that make humans abandon traditional knowledge bases. But what if we told you that there’s more to it?

The Whole Point of a Compiled Wiki

The whole point of a compiled wiki is that the structure does work for you. However, if your graph can’t distinguish ‘this supersedes that’ from ‘this contradicts that,’ you’re leaving some of the most valuable information trapped in unstructured text. This is exactly the problem you were trying to solve.

The Fix: Typed Relationships Inside Wikilinks

To overcome this limitation, we need to introduce typed relationships inside wikilinks. Obsidian-wikilink-types adds semantic relationship types to standard Obsidian wikilinks using the @ syntax. For example, you can use @supersedes to indicate that a note supersedes another one. This allows you to write Dataview queries like “show me everything that contradicts my current hypothesis” and gain a deeper understanding of your knowledge graph.

What This Changes

Insight

With typed links, your vault goes from a tangle of identical connections to a queryable knowledge graph. You can:

Write Dataview queries like “show me everything that contradicts my current hypothesis”
Trace causation chains
See at a glance which notes have been superseded and which are current

This is what Karpathy’s pattern needs but doesn’t have: links that carry meaning.

Gap 2: You Shouldn’t Have to Type Every Relationship Yourself

A wiki with typed links is more useful than one without. However, manually typing every relationship yourself is tedious and prone to errors. The whole premise of the LLM Wiki is that the LLM does the bookkeeping. So, let it discover the relationships too.

The Fix: AI-Discovered Typed Relationships

AI-discovered typed relationships can help alleviate this burden. By leveraging the power of LLMs to identify relationships between notes, you can:

Discover connections that aren’t obvious
Reduce the need for manual typing
Enhance the overall effectiveness of your knowledge graph

The Whole Premise of the LLM Wiki

The whole premise of the LLM Wiki is that the LLM does the bookkeeping. By leveraging AI-discovered typed relationships, you can achieve this goal and create a truly queryable knowledge graph.

Insight

Gap 3: You Shouldn’t Have to Manually Curate Every Note

While AI-discovered typed relationships can help alleviate the burden of manual typing, there’s still a need to manually curate every note in your wiki. This can be a time-consuming and tedious task, especially as your knowledge graph grows.

The Fix: AI-Driven Note Curation

AI-driven note curation can help alleviate this burden. By leveraging the power of LLMs to analyze and categorize notes, you can:

Automatically curate notes based on their content
Reduce the need for manual curation
Enhance the overall effectiveness of your knowledge graph

Conclusion

While Karpathy’s LLM Wiki pattern has brought us closer to the dream of a queryable knowledge graph, it still has its limitations. By introducing typed relationships, AI-discovered typed relationships, and AI-driven note curation, we can overcome these limitations and create a truly effective knowledge graph. It’s time to take the next step and make the most of the LLM Wiki pattern.

Recommendations