Signals: The Core of Reactivity

At the heart of Cortex.jl's reactivity system lies the Signal.

Concept

Think of a Signal as a container for a value that can change over time. The key idea is that other parts of your system can depend on a signal. When the signal's value changes, anything that depends on it (its listeners) is notified, allowing the system to react on changes and recompute values.

Imagine a spreadsheet cell. When you change the value in one cell (like A1), other cells that use A1 in their formulas (like B1 = A1 * 2) automatically update. A Signal is like that cell – it holds a value, and changes can trigger updates elsewhere.

More technically, Signals form a directed graph (potentially cyclic). Each Signal node stores a value, an optional type identifier (UInt8), optional metadata, and maintains lists of its dependencies and listeners. When a signal is updated via set_value!, it propagates a notification to its direct listeners, potentially marking them as 'pending'. This 'pending' state indicates that the signal's value might be stale and needs recomputation. The actual recomputation logic is defined externally via the compute! function. A signal may become 'pending' if all its dependencies meet the criteria: weak dependencies are computed, and strong dependencies are computed and fresh (i.e., have new, unused values). This also means that a signal never recomputes its own value, as it must be done externally.

Key Features

• Value Storage: Holds the current value.
• Type Identifier: Stores an optional UInt8 type (get_type), defaulting to 0x00. This might be particularly useful for choosing different computation strategies for different types of signals within the compute! function.
• Optional Metadata: Can store arbitrary metadata (get_metadata), defaulting to UndefMetadata().
• Dependency Tracking: Knows which other signals it depends on (dependencies) and which signals depend on it (see get_listeners).
• Notification: When updated via set_value!, it notifies its active listeners.
• Pending State: Can be marked as is_pending if its dependencies have updated appropriately, signaling a need for recomputation via compute!.
• External Computation: Relies on the compute! function and a provided strategy to update its value based on dependencies.
• Weak Dependencies: Supports 'weak' dependencies, which influence the pending state based only on whether they are computed (is_computed), not their 'fresh' status in the same way as strong dependencies. See add_dependency! for more details.
• Controlled Listening: Allows dependencies to be added without automatically listening to their updates (listen=false in add_dependency!).

Signal()
Signal(value; type::UInt8 = 0x00, metadata::Any = UndefMetadata())

UndefValue

UndefMetadata

Core Operations

Here are some basic examples demonstrating how to use signals.

Creating Signals and Checking Properties

Signals can be created with or without an initial value. You can optionally specify a type identifier and metadata.

import Cortex

# Create signals
s1 = Cortex.Signal(10)

Signal(value=10, pending=false)

 # Initial value, computed=true
s2 = Cortex.Signal(5)

Signal(value=5, pending=false)

# No initial value, computed=false
s3 = Cortex.Signal()

Signal(value=#undef, pending=false)

# Signal with type and metadata
s4 = Cortex.Signal(true; type=0x01, metadata=Dict(:info => "flag"))

Signal(value=true, pending=false, type=0x01, metadata=Dict(:info => "flag"))

Checking Properties of Signals

Cortex.get_value(s1)   # 10

10

Cortex.get_value(s3)   # Cortex.UndefValue()

Cortex.UndefValue()

Cortex.is_computed(s1) # true

true

Cortex.is_computed(s3) # false

false

Cortex.get_type(s1) # 0x00 (default)

0x00

Cortex.get_type(s4) # 0x01

0x01

Cortex.get_metadata(s1) # UndefMetadata() (default)

Cortex.UndefMetadata()

Cortex.get_metadata(s4) # Dict{Symbol, String}(:info => "flag")

Dict{Symbol, String} with 1 entry:
  :info => "flag"

get_value(s::Signal)

get_type(s::Signal) -> UInt8

get_metadata(s::Signal)

is_computed(s::Signal) -> Bool

is_pending(s::Signal) -> Bool

Adding Dependencies

Signals can depend on other signals. Use add_dependency! to create these links. This populates the dependencies list of the dependent signal and the listeners list of the dependency.

source_1 = Cortex.Signal(1)
source_2 = Cortex.Signal(2)

derived = Cortex.Signal() # A signal that will depend on s1 and s2

Cortex.add_dependency!(derived, source_1)
Cortex.add_dependency!(derived, source_2)

length(Cortex.get_dependencies(derived)) # 2

2

length(Cortex.get_listeners(source_1))           # 1

1

length(Cortex.get_listeners(source_2))           # 1

1

Types of Dependencies

Cortex supports different types of dependencies through options in add_dependency!:

Strong vs. Weak Dependencies

By default, dependencies are "strong," meaning they must be both computed and fresh (recently updated) for a signal to become pending. With weak=true, a dependency only needs to be computed (not necessarily fresh) to contribute to the pending state.

weak_dependency = Cortex.Signal(1)
strong_dependency = Cortex.Signal(2)

derived = Cortex.Signal(3)

Cortex.add_dependency!(derived, weak_dependency; weak=true)
Cortex.add_dependency!(derived, strong_dependency)

Cortex.is_pending(derived) # false

false

Cortex.set_value!(strong_dependency, 10)

Cortex.is_pending(derived) # true

true

Here, even though weak_dependency has not been updated, derived is still in the pending state because it only needs strong_dependency to be updated and weak_dependency only needs to be computed once (or set via constructor).

Listening vs. Non-listening Dependencies

With listen=true (default), a signal is notified when its dependency changes. Setting listen=false creates a dependency relationship without automatic notifications, useful when you want to manually control when a signal responds to changes.

s_source = Cortex.Signal()
s_non_listener = Cortex.Signal()

Cortex.add_dependency!(s_non_listener, s_source; listen=false)

Cortex.is_pending(s_non_listener) # false

false

# Update s_source. s_non_listener is NOT notified.
Cortex.set_value!(s_source, 6)

Cortex.is_pending(s_non_listener) # false

false

Intermediate Dependencies

Setting intermediate=true marks a dependency as intermediate, which affects how process_dependencies! traverses the dependency graph. This is useful for complex dependency trees where some nodes serve as connectors between different parts of the graph.

add_dependency!(signal::Signal, dependency::Signal; weak::Bool = false, listen::Bool = true, intermediate::Bool = false)

get_dependencies(s::Signal) -> Vector{Signal}

get_listeners(s::Signal) -> Vector{Signal}

Pending State

A signal becomes pending (is_pending returns true) when its dependencies are updated in a way that satisfies the pending criteria (all weak dependencies are computed, and all strong dependencies are fresh and computed). Adding a computed dependency can also immediately mark a signal as pending.

Updating a dependency can mark listeners as pending:

source_1 = Cortex.Signal(1)
source_2 = Cortex.Signal(2)

derived = Cortex.Signal() # A signal that will depend on s1 and s2

Cortex.add_dependency!(derived, source_1)
Cortex.add_dependency!(derived, source_2)

Cortex.is_pending(derived) # true

true

derived is pending because both source_1 and source_2 have been computed and are considered fresh with respect to derived at the time of dependency addition. Let's try a different example:

uncomputed_source = Cortex.Signal()
derived = Cortex.Signal()

Cortex.add_dependency!(derived, uncomputed_source)

Cortex.is_pending(derived) # false

false

Cortex.set_value!(uncomputed_source, 1)

Cortex.is_pending(derived) # true

true

After setting the value of uncomputed_source, derived becomes pending because uncomputed_source is now computed and fresh with respect to derived.

Setting Values

Use set_value! to update a signal's value. It also consumes the 'freshness' of its own dependencies and updates the 'fresh' and 'computed' status for its listeners, potentially marking them as pending.

source  = Cortex.Signal(1)
derived = Cortex.Signal()

Cortex.add_dependency!(derived, source)

Cortex.is_pending(derived), Cortex.is_computed(derived)

(true, false)

Cortex.set_value!(derived, 99.0)

Cortex.get_value(derived)   # 99.0

99.0

Cortex.is_pending(derived), Cortex.is_computed(derived) # false, true

(false, true)

set_value!(s::Signal, value::Any)

Computing Signal Values

To compute a signal, use the compute! function, providing a strategy (often a simple function) to calculate the new value based on dependencies. Computing a signal typically clears its pending state.

signal_1 = Cortex.Signal(1)
signal_2 = Cortex.Signal(41)

signal_to_be_computed = Cortex.Signal()

Cortex.add_dependency!(signal_to_be_computed, signal_1)
Cortex.add_dependency!(signal_to_be_computed, signal_2)

Cortex.is_pending(signal_to_be_computed) # true

true

# Define a strategy (a function) to compute the value
compute_sum = (signal, deps) -> sum(Cortex.get_value, deps)

# Apply the strategy using compute!
Cortex.compute!(compute_sum, signal_to_be_computed)

Cortex.get_value(signal_to_be_computed) # 42

42

Cortex.is_pending(signal_to_be_computed) # false

false

# This would normally throw an error:
# compute!(compute_sum, signal_to_be_computed)

# But we can force it:
Cortex.compute!(compute_sum, signal_to_be_computed; force=true)

Cortex.get_value(signal_to_be_computed) # 42

42

Custom Compute Strategies

You can define custom types and methods to implement more complex computation logic beyond simple functions. This allows strategies to hold their own state or parameters.

First, define a struct for your strategy:

struct CustomStrategy
    multiplier::Int
end

Then, implement the compute_value! method for your strategy type:

function Cortex.compute_value!(strategy::CustomStrategy, signal::Cortex.Signal, dependencies)
    # Example: Use signal's metadata if available
    meta = Cortex.get_metadata(signal)
    base_sum = sum(Cortex.get_value, dependencies)
    offset = meta isa Dict && haskey(meta, :offset) ? meta[:offset] : 0
    return strategy.multiplier * base_sum + offset
end

Now, you can use this strategy with your signals:

strategy = CustomStrategy(2)

signal_with_meta = Cortex.Signal(metadata=Dict(:offset => 10))

Cortex.add_dependency!(signal_with_meta, signal_1)
Cortex.add_dependency!(signal_with_meta, signal_2)

Cortex.compute!(strategy, signal_with_meta)

Cortex.get_value(signal_with_meta) # 94

94

Cortex.compute!(CustomStrategy(3), signal_with_meta; force=true)

Cortex.get_value(signal_with_meta) # 136

136

compute!(s::Signal, strategy; force::Bool = false, skip_if_no_listeners::Bool = true)

compute_value!(strategy, signal, dependencies)

Processing Dependency Trees of Signals with process_dependencies!

The process_dependencies! function provides a powerful way to traverse and operate on the dependency graph of a signal. It's particularly useful for scenarios where you need more control over how dependencies are evaluated or when implementing custom update schedulers.

The function recursively applies a user-defined function f to each dependency. f should return true if it considers the dependency "processed" by its own logic, and false otherwise. process_dependencies! propagates this status and can optionally retry processing an intermediate dependency if its own sub-dependencies were processed.

Use Cases:

• Implementing custom evaluation orders for signals.
• Performing actions on dependencies before computing a parent signal.
• Debugging or inspecting the state of a signal's dependency graph.

Conceptual Example:

Imagine you want to traverse the dependency graph of a signal and you want to log which ones were already computed and which ones are not.

signal1 = Cortex.Signal(1; metadata = :signal1) # computed
signal2 = Cortex.Signal(; metadata = :signal2)  # not computed
signal3 = Cortex.Signal(3; metadata = :signal3)  # computed

intermediate_signal = Cortex.Signal(metadata = :intermediate_signal)

Cortex.add_dependency!(intermediate_signal, signal2)
Cortex.add_dependency!(intermediate_signal, signal3)

derived = Cortex.Signal(metadata = :derived)

Cortex.add_dependency!(derived, signal1)
Cortex.add_dependency!(derived, intermediate_signal; intermediate=true)

function my_processing_function(dependency_signal::Cortex.Signal)
    if !Cortex.is_computed(dependency_signal)
        println("The dependency signal: ", Cortex.get_metadata(dependency_signal), " is not computed")
    else
        println("The dependency signal: ", Cortex.get_metadata(dependency_signal), " is computed. The value is: ", Cortex.get_value(dependency_signal))
    end
    # always return false to process all the dependencies
    return false
end

Cortex.process_dependencies!(my_processing_function, derived)

The dependency signal: signal1 is computed. The value is: 1
The dependency signal: intermediate_signal is not computed
The dependency signal: signal2 is not computed
The dependency signal: signal3 is computed. The value is: 3

This example illustrates how you can inject custom logic into the dependency traversal. The actual computation or state change would happen within my_processing_function. For example, here how can we compute! the signal if it is not computed:

signal1 = Cortex.Signal(1; metadata = :signal1)
signal2 = Cortex.Signal(2; metadata = :signal2)
signal3 = Cortex.Signal(3; metadata = :signal3)

intermediate_signal = Cortex.Signal(metadata = :intermediate_signal)

Cortex.add_dependency!(intermediate_signal, signal2)
Cortex.add_dependency!(intermediate_signal, signal3)

derived = Cortex.Signal(metadata = :derived)

Cortex.add_dependency!(derived, signal1)
Cortex.add_dependency!(derived, intermediate_signal; intermediate=true)

function compute_if_not_computed(signal::Cortex.Signal)
    if Cortex.is_pending(signal)
        println("Computing the signal: ", Cortex.get_metadata(signal))
        Cortex.compute!((signal, deps) -> sum(Cortex.get_value, deps), signal)
        return true
    end
    return false
end

Cortex.process_dependencies!(compute_if_not_computed, derived; retry = true)

Computing the signal: intermediate_signal

Now, since we processed and computed all the dependencies, the derived signal should be in the pending state:

Cortex.is_pending(derived) # true

true

Which we can also compute using the same function:

compute_if_not_computed(derived)

Cortex.get_value(derived) # 6

6

process_dependencies!(f::F, signal::Signal; retry::Bool = false) where {F}

Internal Mechanics (For Developers)

The efficient tracking of dependency states (intermediate, weak, computed, and fresh) is managed internally by a structure associated with each signal, SignalDependenciesProps.

SignalDependenciesProps: Packed Dependency Information

To minimize overhead, the properties for each dependency are bit-packed into 4-bit "nibbles" within UInt64 chunks. This allows a single UInt64 to hold status information for 16 dependencies. The bits are assigned as follows (from LSB to MSB):

• Bit 1 (0x1): IsIntermediate: True if the dependency is an intermediate one for processing logic (see process_dependencies!).
• Bit 2 (0x2): IsWeak: True if the dependency is weak.
• Bit 3 (0x4): IsComputed: True if the dependency itself holds a computed value.
• Bit 4 (0x8): IsFresh: True if the dependency has provided a new value that has not yet been consumed by the current signal's computation.

Determining Pending State

The is_pending(signal) function relies on an internal check (currently is_meeting_pending_criteria) that operates on these packed properties. A signal is considered to meet the criteria to become pending if, for every one of its dependencies:

(IsComputed AND (IsWeak OR IsFresh))

This means:

• A weak dependency must simply be IsComputed.
• A strong (non-weak) dependency must be IsComputed AND IsFresh.

When set_value! is called on a signal:

1. The IsFresh flags for all its own dependencies are cleared (as their values have now been "used").
2. For each of its listeners, the original signal (which just got a new value) is marked as IsComputed and IsFresh in that listener's dependency properties. This, in turn, can cause the listener to become pending.

This bit-packed approach allows for efficient batch updates and checks across many dependencies.