Lore — Epic Games' Next-Gen Open Source Version Control System

When Epic Games open-sourced Lore last month, the version control world took notice. Built in Rust from the ground up, Lore is a centralized, content-addressed VCS designed for the workloads that Git struggles with — multi-gigabyte binary files, massive monorepos, thousands of concurrent users, and game development pipelines where artists and developers share the same repository.

At its heart, Lore is two systems: a storage subsystem (a partition-based, content-addressed store that deduplicates all content) and a version control subsystem that builds revisions, branches, merges, and staging out of storage primitives. It's MIT-licensed, already powers Unreal Editor for Fortnite (UEFN), and comes with SDKs for Python, JavaScript, C#, Go, and Rust.

Why Is Lore Trending?

• 3,100+ stars in under a month — rare for an infrastructure project of this ambition
• From Epic Games — the creators of Unreal Engine, Fortnite, and dozens of AAA game franchises
• Rust under the hood — performance, safety, and fearless concurrency from day one
• Binary-first design — Git treats binaries as opaque blobs; Lore chunks them, deduplicates them, and fetches only what you need
• Merkle tree architecture — content-addressed storage with BLAKE3 hashing, immutable revision chains, and cryptographic integrity
• MIT license — fully open source, free for any use
• Multi-language SDKs — bindings for Python, JS, C#, Go, and Rust
• Pre-1.0 but production-proven — already used internally at Epic for UEFN (Unreal Editor for Fortnite)

Architecture Overview

Lore's architecture separates concerns into two clean layers. At the bottom sits the storage subsystem, which provides content-addressed immutable storage (keyed by BLAKE3 hashes) and a separate mutable key-value store for branch pointers and bookkeeping. On top of that, the version control subsystem builds revisions as Merkle trees, with files and directories hashed into a tree of fixed-size nodes.

Data flows through three core primitives:

• Immutable Store — Every piece of content is stored once and addressed by its BLAKE3 hash. Files larger than a configurable threshold are split into chunks using FastCDC (content-defined chunking) or fixed-size splitting, so a single edit inside a multi-gigabyte file re-uploads only the changed chunks.
• Mutable Store — Branch pointers, repository metadata, and name lookups live here. It's a separate, smaller key-value store that supports atomic compare-and-swap operations for safe concurrent updates.
• Shared Store (Local) — On each machine, multiple working instances of the same repository share a single on-disk cache of immutable fragments, reducing disk usage and network transfers.

The Lore Server is a centralized service that manages partitions (16-byte access boundaries enforcing multi-tenant isolation), with optional caching tiers, read replicas, and hot/warm/cold storage backends.

Prerequisites

• A Linux, macOS, or Windows machine (x86_64 or ARM64)
• Ports 41337 (QUIC/gRPC) and 41339 (HTTP) free
• No Rust toolchain needed — prebuilt binaries are available
• No Docker required for the demo mode

Installation & Quickstart

Step 1 — Install Lore in Demo Mode

The install script downloads both the lore CLI and loreserver binary, puts them on your PATH, and starts a local server with an ephemeral store:

curl -fsSL https://raw.githubusercontent.com/EpicGames/lore/main/scripts/install.sh | bash -s -- --demo

On Windows (PowerShell):

$env:LORE_DEMO=1; irm https://raw.githubusercontent.com/EpicGames/lore/main/scripts/install.ps1 | iex

The server runs in this terminal listening on port 41337 (QUIC/gRPC) and 41339 (HTTP). Leave it running and open a new terminal.

Step 2 — Verify Server Health

curl -i http://127.0.0.1:41339/health_check

Expected: HTTP/1.1 200 OK with an empty body. Auth is disabled in demo mode.

Step 3 — Create a Repository

mkdir ~/my-project && cd ~/my-project
lore repository create lore://127.0.0.1:41337/my-project

Lore creates the repository on the server and initializes a working tree with a .lore/ directory containing the client configuration.

Step 4 — Add Files and Stage Them

echo "Hello, Lore" > hello.txt
python3 -c "import os; open('sample.bin', 'wb').write(os.urandom(256))"
lore add .

The lore add command stages files — unlike Git, Lore doesn't materialize fragments locally until commit time.

Step 5 — Commit

lore commit -m "Initial commit"

This creates a revision with a Merkle tree of the directory structure, hashes the file contents (chunking large files automatically), and stores everything in the immutable store.

Step 6 — Push to Server

lore push

Lore's push protocol uses resumable transfer — if the connection drops mid-push, it picks up where it left off. Only new fragments are sent.

Step 7 — Clone Into a Second Working Tree

mkdir ~/my-project-clone && cd ~/my-project-clone
lore clone lore://127.0.0.1:41337/my-project

Both working trees share the same on-disk shared store, so fragments are downloaded only once per machine.

Step 8 — Branching and Merging

Create a branch, make changes, and merge back:

lore branch feature-1
echo "Feature work" > feature.txt
lore add .
lore commit -m "Add feature"
lore checkout main
lore merge feature-1
lore push

Branches are lightweight mutable references — no data duplication. Merges use the Merkle tree structure to compute the merge base efficiently.

Key Concepts

Content-Addressed Storage

Every byte in Lore is addressed by its BLAKE3 hash. This means:

• Identical files across branches or directories are stored once
• Integrity is verified by hashing — if the hash matches, the content is correct
• Comparisons between revisions are fast hash lookups, not byte-by-byte scans

Chunked Large File Storage

Files above a threshold (default configurable) are split into chunks using FastCDC (content-defined chunking) — boundaries are determined by the content itself, not fixed byte offsets. This means:

• Editing 10 bytes in the middle of a 10 GB file uploads only the 2-3 affected chunks
• Any byte range can be read without materializing the entire file
• Deduplication works across versions even when insertions shift byte boundaries

Immutable Revision Chain

Each revision's hash is derived from its state (file tree hashes, parent revisions, metadata), forming a tamper-evident chain. You cannot rewrite history without breaking the chain — and the chain breakage is cryptographically detectable.

Sparse Working Copies by Default

Lore doesn't download everything upfront. Working trees fetch file data on demand as you access files. This is critical for large repositories where a full checkout would take hours or require terabytes of local storage.

Comparison: Lore vs Git vs Perforce

• Binary files: Git treats binaries as opaque blobs (huge .git directory). Lore chunks them, deduplicates across versions, and fetches only needed ranges.
• Scalability: Git bogs down at 50,000+ files or 100+ GB repos. Lore is designed for millions of files and multi-TB repositories.
• Concurrent users: Git's distributed model doesn't enforce access control. Lore is centralized with partition-based multi-tenancy and fine-grained auth.
• Atomic operations: Git has no server-side atomicity for pushes. Lore uses compare-and-swap on the mutable store for safe concurrent updates.
• Checkout speed: Git clones download everything. Lore's sparse working copies fetch lazily — initial clone is seconds, not hours.
• Large file handling: Git LFS is a bolted-on extension. Lore handles large files natively with chunking, FastCDC, and sparse reads.
• API surface: Git's internals are porcelain/plumbing with evolving conventions. Lore has a versioned, public API with SDKs for 5 languages.

Verification Checklist

• Lore server starts and responds to health checks on port 41339
• Repository creation succeeds with lore repository create
• Files can be added, committed, and pushed
• Cloning produces a working second tree
• Branching and merging produce correct revision history
• Large binary files are chunked and deduplicated

Resources

• GitHub Repository: github.com/EpicGames/lore
• Official Documentation: epicgames.github.io/lore/
• Quickstart Guide: epicgames.github.io/lore/tutorials/quickstart/
• System Design Doc: epicgames.github.io/lore/explanation/system-design/
• Roadmap: epicgames.github.io/lore/roadmap/
• FAQ: epicgames.github.io/lore/faq/
• Discord Community: discord.gg/E4SFJKRPbg
• Python SDK: github.com/EpicGames/lore-python
• JavaScript SDK: github.com/EpicGames/lore-js
• C# SDK: github.com/EpicGames/lore-dotnet
• Go SDK: github.com/EpicGames/lore-go