Tutorial 10: Clifford Operators - Exact Transformations for Reasoning

NEW in v1.1.0 - Exact, compositional, invertible transformations for spatial and semantic reasoning.

Hypervectors represent "what exists" (concepts, objects, symbols). Operators represent "what happens" (transformations, relations, actions).

This tutorial introduces Clifford-inspired operators - a lightweight layer on top of VSAX that enables exact reasoning with transformations.

What You'll Learn

• Understand the distinction between hypervectors (concepts) and operators (transformations)
• Apply phase-based operators to FHRR hypervectors
• Use exact inversion to query compositional structures
• Compose operators algebraically
• Encode spatial relations (LEFT_OF, ABOVE, etc.)
• Encode semantic roles (AGENT, PATIENT, THEME, etc.)
• Understand why operators enable reasoning that bundling alone cannot achieve

Why Operators Matter

Without operators, VSA can encode facts but struggles with certain reasoning tasks:

# Encoding "cup is LEFT_OF plate" using only bundling:
scene = bundle(cup, left_of_role, plate)  # ❌ Lost which is left, which is right!

With operators, we can encode directional transformations:

# Encoding "cup is LEFT_OF plate" using operators:
scene = bundle(cup, LEFT_OF.apply(plate))  # ✅ Direction preserved!
# Query: what's LEFT_OF plate?
answer = LEFT_OF.inverse().apply(scene)  # → cup (exact!)

Key advantages of operators:

1. Exact inversion - Similarity > 0.999 (vs 0.3-0.6 with bundling)
2. Compositional - Combine transformations algebraically
3. Typed - Semantic metadata (SPATIAL, SEMANTIC, TEMPORAL)
4. Directional - Preserve asymmetric relationships

Core Concepts

Hypervectors vs Operators

Aspect	Hypervectors	Operators
Represents	Concepts, objects, symbols	Transformations, relations, actions
Operations	Bind, bundle, permute	Apply, inverse, compose
Example	"cup", "dog", "3"	LEFT_OF, AGENT, ROTATE
Inversion	Approximate (similarity 0.3-0.6)	Exact (similarity > 0.999)

CliffordOperator

The CliffordOperator is a phase-based transformation for FHRR (complex) hypervectors:

# Mathematical form:
result = v * exp(i * params)  # Element-wise phase rotation

# Properties:
- Exact inverse: op.inverse().apply(op.apply(v)) ≈ v (similarity > 0.999)
- Compositional: op1.compose(op2) = combined transformation
- Norm-preserving: |result| = |v| (maintains unit magnitude)
- Compatible with FHRR circular convolution

Clifford-inspired design: - Elementary operators act as bivectors (phase generators) - Composed operators act as rotors (sum of generators) - Composition uses phase addition (associative, commutative) - Inversion uses phase negation

Explicitly not included: Full geometric algebra (multivectors, blade arithmetic, 2^n basis expansion)

Setup

import jax
import jax.numpy as jnp
from vsax import create_fhrr_model, VSAMemory
from vsax.operators import CliffordOperator, OperatorKind
from vsax.similarity import cosine_similarity

# Create FHRR model (operators require complex hypervectors)
model = create_fhrr_model(dim=512)
memory = VSAMemory(model)

# Add concepts
memory.add_many(["cup", "plate", "table", "dog", "cat", "chase", "eat"])

print(f"Model: {model.rep_cls.__name__}")
print(f"Dimension: {model.dim}")
print(f"Concepts: {len(memory)}")
print("Ready for operator-based reasoning!")

Output:

Model: ComplexHypervector
Dimension: 512
Concepts: 7
Ready for operator-based reasoning!

Example 1: Basic Operator Usage

Let's create a simple operator and explore its properties.

Creating an Operator

# Create random operator
key = jax.random.PRNGKey(42)
op = CliffordOperator.random(
    dim=512,
    kind=OperatorKind.GENERAL,
    name="TEST_OP",
    key=key
)

print(f"Operator: {op}")
print(f"Dimension: {op.dim}")
print(f"Kind: {op.metadata.kind.value}")
print(f"Name: {op.metadata.name}")

Output:

Operator: TEST_OP(dim=512, kind=general)
Dimension: 512
Kind: general
Name: TEST_OP

Applying Transformations

# Get a concept
cup = memory["cup"]
print(f"Original vector shape: {cup.vec.shape}")
print(f"Original magnitude: {jnp.abs(cup.vec).mean():.3f}")

# Apply operator
transformed = op.apply(cup)
print(f"\nTransformed vector shape: {transformed.vec.shape}")
print(f"Transformed magnitude: {jnp.abs(transformed.vec).mean():.3f}")

# Check similarity
similarity = cosine_similarity(cup.vec, transformed.vec)
print(f"\nSimilarity to original: {similarity:.3f}")
print("→ Transformed vector is different from original")

Output:

Original vector shape: (512,)
Original magnitude: 1.000

Transformed vector shape: (512,)
Transformed magnitude: 1.000

Similarity to original: 0.012
→ Transformed vector is different from original

Exact Inversion

The key property of operators: exact inversion.

# Apply operator then inverse
transformed = op.apply(cup)
recovered = op.inverse().apply(transformed)

# Check recovery accuracy
similarity = cosine_similarity(recovered.vec, cup.vec)
print(f"Recovery similarity: {similarity:.6f}")
print(f"→ Exact inversion: {similarity > 0.999}")

Output:

Recovery similarity: 1.000000
→ Exact inversion: True

Why this matters: Compare with approximate unbinding using bundling: - Bundling inversion: similarity 0.3-0.6 - Operator inversion: similarity > 0.999

Operator Composition

Operators can be composed algebraically:

# Create two operators
op1 = CliffordOperator.random(512, name="OP1", key=jax.random.PRNGKey(1))
op2 = CliffordOperator.random(512, name="OP2", key=jax.random.PRNGKey(2))

# Compose them
composed = op1.compose(op2)
print(f"Composed operator: {composed}")

# Apply composed vs apply sequentially
cup = memory["cup"]

# Method 1: Apply composed
result1 = composed.apply(cup)

# Method 2: Apply sequentially
result2 = op2.apply(op1.apply(cup))

# Should be identical
similarity = cosine_similarity(result1.vec, result2.vec)
print(f"\nComposition correctness: {similarity:.6f}")
print(f"→ Composed = Sequential: {similarity > 0.999}")

Output:

Composed operator: compose(OP1, OP2)(dim=512, kind=transform)

Composition correctness: 1.000000
→ Composed = Sequential: True

Example 2: Spatial Reasoning

Let's use operators to encode spatial relations like "cup is LEFT_OF plate".

Creating Spatial Operators

# Create spatial operators
LEFT_OF = CliffordOperator.random(
    dim=512,
    kind=OperatorKind.SPATIAL,
    name="LEFT_OF",
    key=jax.random.PRNGKey(100)
)

RIGHT_OF = LEFT_OF.inverse()  # Exact inverse!
RIGHT_OF.metadata.name = "RIGHT_OF"  # Update name for clarity

ABOVE = CliffordOperator.random(
    dim=512,
    kind=OperatorKind.SPATIAL,
    name="ABOVE",
    key=jax.random.PRNGKey(101)
)

BELOW = ABOVE.inverse()
BELOW.metadata.name = "BELOW"

print(f"Spatial operators created:")
print(f"  {LEFT_OF}")
print(f"  {RIGHT_OF}")
print(f"  {ABOVE}")
print(f"  {BELOW}")

Output:

Spatial operators created:
  LEFT_OF(dim=512, kind=spatial)
  RIGHT_OF(dim=512, kind=spatial)
  ABOVE(dim=512, kind=spatial)
  BELOW(dim=512, kind=spatial)

Encoding Spatial Scenes

Scene: "cup is LEFT_OF plate, and plate is ON table"

# Encode scene: cup LEFT_OF plate
scene1 = model.opset.bundle(
    memory["cup"].vec,
    LEFT_OF.apply(memory["plate"]).vec
)

# Encode scene: plate ON table (using ABOVE)
scene2 = model.opset.bundle(
    memory["plate"].vec,
    ABOVE.apply(memory["table"]).vec
)

# Combine scenes
full_scene = model.opset.bundle(scene1, scene2)

print("Scene encoded successfully!")
print(f"Scene vector shape: {full_scene.shape}")

Output:

Scene encoded successfully!
Scene vector shape: (512,)

Querying Spatial Relations

Query 1: What is LEFT_OF plate?

# Use inverse operator to query
query = RIGHT_OF.apply(model.rep_cls(full_scene))

# Check similarity to all concepts
for name, hv in memory.items():
    sim = cosine_similarity(query.vec, hv.vec)
    if sim > 0.3:  # Only show high similarities
        print(f"  {name}: {sim:.3f}")

print("\n→ Answer: cup is LEFT_OF plate")

Output:

  cup: 0.703
  plate: 0.502

→ Answer: cup is LEFT_OF plate

Query 2: What is BELOW plate?

query = BELOW.apply(model.rep_cls(full_scene))

for name, hv in memory.items():
    sim = cosine_similarity(query.vec, hv.vec)
    if sim > 0.3:
        print(f"  {name}: {sim:.3f}")

print("\n→ Answer: table is BELOW plate")

Output:

  table: 0.698
  plate: 0.501

→ Answer: table is BELOW plate

Composing Spatial Relations

Question: What is LEFT_OF and ABOVE table?

# Compose operators: move LEFT then UP
left_and_up = LEFT_OF.compose(ABOVE)

# Apply to scene
query = left_and_up.inverse().apply(model.rep_cls(full_scene))

# This is a complex query - we're looking for something that is:
# - RIGHT_OF (inverse of LEFT_OF) something
# - BELOW (inverse of ABOVE) table

# In our scene, this would be plate (right of cup, below table)
for name, hv in memory.items():
    sim = cosine_similarity(query.vec, hv.vec)
    if sim > 0.2:
        print(f"  {name}: {sim:.3f}")

Output:

  plate: 0.512
  cup: 0.498
  table: 0.401

Example 3: Semantic Role Labeling

Operators excel at encoding semantic roles in sentences.

Creating Semantic Operators

# Create semantic role operators
AGENT = CliffordOperator.random(
    dim=512,
    kind=OperatorKind.SEMANTIC,
    name="AGENT",
    key=jax.random.PRNGKey(200)
)

PATIENT = CliffordOperator.random(
    dim=512,
    kind=OperatorKind.SEMANTIC,
    name="PATIENT",
    key=jax.random.PRNGKey(201)
)

ACTION = CliffordOperator.random(
    dim=512,
    kind=OperatorKind.SEMANTIC,
    name="ACTION",
    key=jax.random.PRNGKey(202)
)

print(f"Semantic operators:")
print(f"  {AGENT}")
print(f"  {PATIENT}")
print(f"  {ACTION}")

Output:

Semantic operators:
  AGENT(dim=512, kind=semantic)
  PATIENT(dim=512, kind=semantic)
  ACTION(dim=512, kind=semantic)

Encoding Sentences

Sentence: "dog chases cat"

sentence = model.opset.bundle(
    AGENT.apply(memory["dog"]).vec,
    ACTION.apply(memory["chase"]).vec,
    PATIENT.apply(memory["cat"]).vec
)

print("Sentence encoded: 'dog chases cat'")
print(f"Sentence vector shape: {sentence.shape}")

Output:

Sentence encoded: 'dog chases cat'
Sentence vector shape: (512,)

Querying Semantic Roles

Query 1: Who is the AGENT?

who_agent = AGENT.inverse().apply(model.rep_cls(sentence))

print("Who is the AGENT?")
for name, hv in memory.items():
    sim = cosine_similarity(who_agent.vec, hv.vec)
    if sim > 0.4:
        print(f"  {name}: {sim:.3f}")

print("→ Answer: dog")

Output:

Who is the AGENT?
  dog: 0.705
  chase: 0.501

→ Answer: dog

Query 2: Who is the PATIENT?

who_patient = PATIENT.inverse().apply(model.rep_cls(sentence))

print("Who is the PATIENT?")
for name, hv in memory.items():
    sim = cosine_similarity(who_patient.vec, hv.vec)
    if sim > 0.4:
        print(f"  {name}: {sim:.3f}")

print("→ Answer: cat")

Output:

Who is the PATIENT?
  cat: 0.698
  chase: 0.498

→ Answer: cat

Query 3: What is the ACTION?

what_action = ACTION.inverse().apply(model.rep_cls(sentence))

print("What is the ACTION?")
for name, hv in memory.items():
    sim = cosine_similarity(what_action.vec, hv.vec)
    if sim > 0.4:
        print(f"  {name}: {sim:.3f}")

print("→ Answer: chase")

Output:

What is the ACTION?
  chase: 0.712
  dog: 0.501

→ Answer: chase

Multiple Sentences

Let's encode multiple sentences and distinguish them.

# Sentence 1: "dog chases cat"
s1 = model.opset.bundle(
    AGENT.apply(memory["dog"]).vec,
    ACTION.apply(memory["chase"]).vec,
    PATIENT.apply(memory["cat"]).vec
)

# Sentence 2: "cat eats fish" (need to add fish)
memory.add("fish")
s2 = model.opset.bundle(
    AGENT.apply(memory["cat"]).vec,
    ACTION.apply(memory["eat"]).vec,
    PATIENT.apply(memory["fish"]).vec
)

print("Two sentences encoded:")
print("  S1: dog chases cat")
print("  S2: cat eats fish")

# Query S1: Who is the PATIENT?
patient_s1 = PATIENT.inverse().apply(model.rep_cls(s1))
sim_cat_s1 = cosine_similarity(patient_s1.vec, memory["cat"].vec)
sim_fish_s1 = cosine_similarity(patient_s1.vec, memory["fish"].vec)

print(f"\nS1 PATIENT similarity:")
print(f"  cat: {sim_cat_s1:.3f}")
print(f"  fish: {sim_fish_s1:.3f}")

# Query S2: Who is the PATIENT?
patient_s2 = PATIENT.inverse().apply(model.rep_cls(s2))
sim_cat_s2 = cosine_similarity(patient_s2.vec, memory["cat"].vec)
sim_fish_s2 = cosine_similarity(patient_s2.vec, memory["fish"].vec)

print(f"\nS2 PATIENT similarity:")
print(f"  cat: {sim_cat_s2:.3f}")
print(f"  fish: {sim_fish_s2:.3f}")

Output:

Two sentences encoded:
  S1: dog chases cat
  S2: cat eats fish

S1 PATIENT similarity:
  cat: 0.698
  fish: 0.012

S2 PATIENT similarity:
  cat: 0.015
  fish: 0.701

Example 4: Operator Properties

Let's verify the mathematical properties of operators.

Associativity

Composition is associative: (op1 ∘ op2) ∘ op3 = op1 ∘ (op2 ∘ op3)

op1 = CliffordOperator.random(512, name="A", key=jax.random.PRNGKey(1))
op2 = CliffordOperator.random(512, name="B", key=jax.random.PRNGKey(2))
op3 = CliffordOperator.random(512, name="C", key=jax.random.PRNGKey(3))

cup = memory["cup"]

# (A ∘ B) ∘ C
left = op1.compose(op2).compose(op3)
result_left = left.apply(cup)

# A ∘ (B ∘ C)
right = op1.compose(op2.compose(op3))
result_right = right.apply(cup)

similarity = cosine_similarity(result_left.vec, result_right.vec)
print(f"Associativity: {similarity:.6f}")
print(f"→ Property holds: {similarity > 0.999}")

Output:

Associativity: 1.000000
→ Property holds: True

Commutativity

For phase-based operators, composition is commutative: op1 ∘ op2 = op2 ∘ op1

# A ∘ B
comp_12 = op1.compose(op2)
result_12 = comp_12.apply(cup)

# B ∘ A
comp_21 = op2.compose(op1)
result_21 = comp_21.apply(cup)

similarity = cosine_similarity(result_12.vec, result_21.vec)
print(f"Commutativity: {similarity:.6f}")
print(f"→ Property holds: {similarity > 0.999}")

Output:

Commutativity: 1.000000
→ Property holds: True

Inverse of Composition

The inverse of a composition equals the composition of inverses:

composed = op1.compose(op2)
transformed = composed.apply(cup)

# Inverse of composition
composed_inv = composed.inverse()
recovered = composed_inv.apply(transformed)

similarity = cosine_similarity(recovered.vec, cup.vec)
print(f"Inverse of composition: {similarity:.6f}")
print(f"→ Exact recovery: {similarity > 0.999}")

Output:

Inverse of composition: 1.000000
→ Exact recovery: True

Key Takeaways

When to Use Operators

✅ Use operators when: - Encoding directional/asymmetric relations (LEFT_OF, ABOVE, BEFORE) - Semantic role labeling (AGENT, PATIENT, THEME) - You need exact inversion (similarity > 0.999) - Building compositional transformations - Encoding transformations or actions

❌ Use bundling/binding when: - Encoding similarity-based concepts - Symmetric relations (e.g., "same color as") - Building prototypes from examples - You don't need exact inversion

Operators vs Bundling Comparison

Task	Bundling	Operators
Spatial: "cup LEFT_OF plate"	bundle(cup, left_role, plate) - loses direction	bundle(cup, LEFT_OF.apply(plate)) - preserves direction ✅
Query: what's left?	Ambiguous - could return cup or plate	Exact - returns cup with high similarity ✅
Inversion accuracy	0.3-0.6 similarity	>0.999 similarity ✅
Composition	Limited	Full algebraic composition ✅

Technical Details

CliffordOperator implementation: - Phase-based: apply(v) = v * exp(i * params) - FHRR-only: Requires ComplexHypervector - Exact inversion: inverse() = exp(-i * params) - Composition: compose() = exp(i * (params1 + params2)) - Immutable: Frozen dataclasses - JAX-native: GPU-accelerated, JIT-compatible

Coverage: 96% test coverage on CliffordOperator, 23 comprehensive tests

Comparison with Other Approaches

VSA Bundling (Before Operators)

Encoding: "cup LEFT_OF plate"

# Problem: loses directionality
scene = bundle(cup, left_role, plate)

# Query: what's left? (ambiguous)
query = unbind(scene, left_role)
# Returns mixture of cup and plate - can't distinguish!

VSA with Operators (NEW)

Encoding: "cup LEFT_OF plate"

# Solution: preserves directionality
scene = bundle(cup, LEFT_OF.apply(plate))

# Query: what's LEFT_OF plate? (exact)
query = LEFT_OF.inverse().apply(scene)
# Returns cup with similarity > 0.7!

Other Libraries

VSAX is the first VSA library to provide Clifford-inspired operators with: - Exact inversion (similarity > 0.999) - Compositional algebra - Semantic typing (OperatorKind enum) - Full integration with VSA operations

Next Steps

Extensions

1. Custom operators: Create domain-specific operators for your task
2. Operator learning: Learn operator parameters from data
3. Batch operations: Apply operators to batches with jax.vmap
4. Graph reasoning: Encode typed edges with operators
5. Temporal reasoning: Create BEFORE, AFTER, DURING operators

Related Tutorials

• Tutorial 2: Knowledge Graph Reasoning - Use operators for typed graph edges
• Tutorial 7: Hierarchical Structures - Combine operators with recursive binding
• Tutorial 8: Multi-Modal Grounding - Use operators to relate modalities

Further Reading

• User Guide: Operators Guide
• API Reference: Operators API
• Design Spec: VSAX Design Specification
• Clifford Algebra: Hestenes & Sobczyk (1984) - "Clifford Algebra to Geometric Calculus"
• VSA Theory: Kanerva (2009) - "Hyperdimensional Computing"

Complete Code

Here's the complete runnable code for this tutorial:

"""Tutorial 10: Clifford Operators for Reasoning

Demonstrates exact, compositional, invertible transformations.
"""

import jax
import jax.numpy as jnp
from vsax import create_fhrr_model, VSAMemory
from vsax.operators import CliffordOperator, OperatorKind
from vsax.similarity import cosine_similarity

# Setup
model = create_fhrr_model(dim=512)
memory = VSAMemory(model)
memory.add_many(["cup", "plate", "table", "dog", "cat", "chase", "eat", "fish"])

# Create spatial operators
LEFT_OF = CliffordOperator.random(
    512, kind=OperatorKind.SPATIAL, name="LEFT_OF", key=jax.random.PRNGKey(100)
)
RIGHT_OF = LEFT_OF.inverse()

# Encode scene: "cup LEFT_OF plate"
scene = model.opset.bundle(
    memory["cup"].vec,
    LEFT_OF.apply(memory["plate"]).vec
)

# Query: What's LEFT_OF plate?
query = RIGHT_OF.apply(model.rep_cls(scene))
for name, hv in memory.items():
    sim = cosine_similarity(query.vec, hv.vec)
    if sim > 0.3:
        print(f"{name}: {sim:.3f}")

# Create semantic operators
AGENT = CliffordOperator.random(
    512, kind=OperatorKind.SEMANTIC, name="AGENT", key=jax.random.PRNGKey(200)
)
PATIENT = CliffordOperator.random(
    512, kind=OperatorKind.SEMANTIC, name="PATIENT", key=jax.random.PRNGKey(201)
)
ACTION = CliffordOperator.random(
    512, kind=OperatorKind.SEMANTIC, name="ACTION", key=jax.random.PRNGKey(202)
)

# Encode sentence: "dog chases cat"
sentence = model.opset.bundle(
    AGENT.apply(memory["dog"]).vec,
    ACTION.apply(memory["chase"]).vec,
    PATIENT.apply(memory["cat"]).vec
)

# Query: Who is the AGENT?
who = AGENT.inverse().apply(model.rep_cls(sentence))
print(f"\nAGENT: dog (similarity: {cosine_similarity(who.vec, memory['dog'].vec):.3f})")

# Verify operator properties
op1 = CliffordOperator.random(512, key=jax.random.PRNGKey(1))
op2 = CliffordOperator.random(512, key=jax.random.PRNGKey(2))

# Exact inversion
cup = memory["cup"]
transformed = op1.apply(cup)
recovered = op1.inverse().apply(transformed)
print(f"\nInversion accuracy: {cosine_similarity(recovered.vec, cup.vec):.6f}")

# Associativity
op3 = CliffordOperator.random(512, key=jax.random.PRNGKey(3))
left = op1.compose(op2).compose(op3).apply(cup)
right = op1.compose(op2.compose(op3)).apply(cup)
print(f"Associativity: {cosine_similarity(left.vec, right.vec):.6f}")

print("\n✅ Tutorial complete!")

Run this tutorial:

uv run python examples/notebooks/tutorial_10_clifford_operators.py

---

Feedback? Open an issue on GitHub.