Instance Methods

class pydtmc.MarkovChain[source]
absorption_probabilities()[source]

The method computes the absorption probabilities of the Markov chain.

Notes:

  • If the Markov chain has no transient states, then None is returned.

Return Type:

Optional[ndarray]

aggregate(s, method='adaptive')[source]

The method attempts to reduce the state space of the Markov chain to the given number of states through a Kullback-Leibler divergence minimization approach.

Notes:
Parameters:

  • s (int) – the number of states of the reduced Markov chain.

  • method (str)

    • spectral-bottom-up for a spectral theory based aggregation, bottom-up (more suitable for reducing a large number of states).

    • spectral-top-down for a spectral theory based aggregation, top-down (more suitable for reducing a small number of states).

    • adaptive for automatically selecting the best aggregation method.

Raises:

  • ValidationError – if any input argument is not compliant.

  • ValueError – if the Markov chain defines only two states or is not ergodic.

Return Type:

MarkovChain

are_communicating(state1, state2)[source]

The method verifies whether the given states of the Markov chain are communicating.

Parameters:
Raises:

ValidationError – if any input argument is not compliant.

Return Type:

bool

closest_reversible(initial_distribution=None, weighted=False)[source]

The method computes the closest reversible of the Markov chain.

Notes:
Parameters:

  • initial_distribution (Optional[Union [ndarray, spmatrix]]) – the distribution of the states (if omitted, the states are assumed to be uniformly distributed).

  • weighted (bool) – a boolean indicating whether to use the weighted Frobenius norm.

Raises:

  • ValidationError – if any input argument is not compliant.

  • ValueError – if the closest reversible could not be computed.

Return Type:

MarkovChain

committor_probabilities(committor_type, states1, states2)[source]

The method computes the committor probabilities between the given subsets of the state space defined by the Markov chain.

Notes:

  • If the Markov chain is not ergodic, then None is returned.

  • The method can be accessed through the following aliases: crp.

Parameters:

  • committor_type (str)

    • backward for the backward committor;

    • forward for the forward committor.

  • states1 (Union[int, str, List[int], List[str]]) – the first subset of the state space.

  • states2 (Union[int, str, List[int], List[str]]) – the second subset of the state space.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Optional[ndarray]

conditional_probabilities(state)[source]

The method computes the probabilities, for all the states of the Markov chain, conditioned on the process being at the given state.

Notes:

  • The method can be accessed through the following aliases: conditional_distribution, cd, cp.

Parameters:

state (Union[int, str]) – the current state.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

ndarray

expected_rewards(steps, rewards)[source]

The method computes the expected rewards of the Markov chain after N steps, given the reward value of each state.

Notes:

  • The method can be accessed through the following aliases: er.

Parameters:
Raises:

ValidationError – if any input argument is not compliant.

Return Type:

ndarray

expected_transitions(steps, initial_distribution=None)[source]

The method computes the expected number of transitions performed by the Markov chain after N steps, given the initial distribution of the states.

Notes:

  • The method can be accessed through the following aliases: et.

Parameters:

  • steps (int) – the number of steps.

  • initial_distribution (Optional[Union [ndarray, spmatrix]]) – the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

ndarray

first_passage_probabilities(steps, initial_state, first_passage_states=None)[source]

The method computes the first passage probabilities of the Markov chain after N steps, given the initial state and, optionally, the first passage states.

Notes:

  • The method can be accessed through the following aliases: fpp.

Parameters:
Raises:

ValidationError – if any input argument is not compliant.

Return Type:

ndarray

first_passage_reward(steps, initial_state, first_passage_states, rewards)[source]

The method computes the first passage reward of the Markov chain after N steps, given the reward value of each state, the initial state and the first passage states.

Notes:

  • The method can be accessed through the following aliases: fpr.

Parameters:
Raises:
Return Type:

float

hitting_probabilities(targets=None)[source]

The method computes the hitting probability, for the states of the Markov chain, to the given set of states.

Notes:

  • The method can be accessed through the following aliases: hp.

Parameters:

targets (Optional[Union [int, str, List[int], List[str]]]) – the target states (if omitted, all the states are targeted).

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

ndarray

hitting_times(targets=None)[source]

The method computes the hitting times, for all the states of the Markov chain, to the given set of states.

Notes:

  • The method can be accessed through the following aliases: ht.

Parameters:

targets (Optional[Union [int, str, List[int], List[str]]]) – the target states (if omitted, all the states are targeted).

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

ndarray

is_absorbing_state(state)[source]

The method verifies whether the given state of the Markov chain is absorbing.

Parameters:

state (Union[int, str]) – the target state.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

bool

is_accessible(state_target, state_origin)[source]

The method verifies whether the given target state is reachable from the given origin state.

Parameters:
Raises:

ValidationError – if any input argument is not compliant.

Return Type:

bool

is_cyclic_state(state)[source]

The method verifies whether the given state is cyclic.

Parameters:

state (Union[int, str]) – the target state.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

bool

is_recurrent_state(state)[source]

The method verifies whether the given state is recurrent.

Parameters:

state (Union[int, str]) – the target state.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

bool

is_transient_state(state)[source]

The method verifies whether the given state is transient.

Parameters:

state (Union[int, str]) – the target state.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

bool

lump(partitions)[source]

The method attempts to reduce the state space of the Markov chain with respect to the given partitions following the ordinary lumpability criterion.

Parameters:

partitions (Union[List[List[int]], List[List[str]]]) – the partitions of the state space.

Raises:

  • ValidationError – if any input argument is not compliant.

  • ValueError – if the Markov chain defines only two states or is not lumpable with respect to the given partitions.

Return Type:

MarkovChain

mean_absorption_times()[source]

The method computes the mean absorption times of the Markov chain.

Notes:

  • If the Markov chain is not absorbing or has no transient states, then None is returned.

  • The method can be accessed through the following aliases: mat.

Return Type:

Optional[ndarray]

mean_first_passage_times_between(origins, targets)[source]

The method computes the mean first passage times between the given subsets of the state space.

Notes:

  • If the Markov chain is not ergodic, then None is returned.

  • The method can be accessed through the following aliases: mfpt_between, mfptb.

Parameters:
Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Optional[float]

mean_first_passage_times_to(targets=None)[source]

The method computes the mean first passage times, for all the states, to the given set of states.

Notes:

  • If the Markov chain is not ergodic, then None is returned.

  • The method can be accessed through the following aliases: mfpt_to, mfptt.

Parameters:

targets (Optional[Union [int, str, List[int], List[str]]]) – the target states (if omitted, all the states are targeted).

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Optional[ndarray]

mean_number_visits()[source]

The method computes the mean number of visits of the Markov chain.

Notes:

  • The method can be accessed through the following aliases: mnv.

Return Type:

Optional[ndarray]

mean_recurrence_times()[source]

The method computes the mean recurrence times of the Markov chain.

Notes:

  • If the Markov chain is not ergodic, then None is returned.

  • The method can be accessed through the following aliases: mrt.

Return Type:

Optional[ndarray]

merge_with(other, gamma)[source]

The method returns a Markov chain whose transition matrix is defined below.

\(p_{new} = (1 - \gamma) p_{current} + \gamma p_{other}\)
Parameters:

  • other (MarkovChain) – the other Markov chain to be merged.

  • gamma (float) – the merger blending factor.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

MarkovChain

mixing_time(initial_distribution=None, jump=1, cutoff_type='natural')[source]

The method computes the mixing time of the Markov chain, given the initial distribution of the states.

Notes:

  • If the Markov chain is not ergodic, then None is returned.

  • The method can be accessed through the following aliases: mt.

Parameters:

  • initial_distribution (Optional[Union [ndarray, spmatrix]]) – the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).

  • jump (int) – the number of steps in each iteration.

  • cutoff_type (str)

    • natural for the natural cutoff;

    • traditional for the traditional cutoff.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Optional[int]

next(initial_state, output_index=False, seed=None)[source]

The method simulates a single step in a random walk.

Parameters:

  • initial_state (Union[int, str]) – the initial state.

  • output_index (bool) – a boolean indicating whether to output the state index.

  • seed (Optional[int]) – a seed to be used as RNG initializer for reproducibility purposes.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Union[int, str]

predict(steps, initial_state, output_indices=False)[source]

The method computes the most probable sequence of states produced by a random walk of N steps, given the initial state.

Notes:

  • In presence of probability ties None is returned.

Parameters:

  • steps (int) – the number of steps.

  • initial_state (Union[int, str]) – the initial state.

  • output_indices (bool) – a boolean indicating whether to output the state indices.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Optional[Union [List[int], List[str]]]

redistribute(steps, initial_status=None, output_last=True)[source]

The method performs a redistribution of states of N steps.

Parameters:

  • steps (int) – the number of steps.

  • initial_status (Optional[Union [int, str, ndarray, spmatrix]]) – the initial state or the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).

  • output_last (bool) – a boolean indicating whether to output only the last distributions.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Union[ndarray, List[ndarray]]

sensitivity(state)[source]

The method computes the sensitivity matrix of the stationary distribution with respect to a given state.

Notes:

  • If the Markov chain is not irreducible, then None is returned.

Parameters:

state (Union[int, str]) – the target state.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Optional[ndarray]

sequence_probability(sequence)[source]

The method computes the probability of a given sequence of states.

Parameters:

sequence (Union[List[int], List[str]]) – the observed sequence of states.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

float

simulate(steps, initial_state=None, final_state=None, output_indices=False, seed=None)[source]

The method simulates a random walk of the given number of steps.

Parameters:

  • steps (int) – the number of steps.

  • initial_state (Optional[Union [int, str]]) – the initial state (if omitted, it is chosen uniformly at random).

  • final_state (Optional[Union [int, str]]) – the final state of the walk (if specified, the simulation stops as soon as it is reached even if not all the steps have been performed).

  • output_indices (bool) – a boolean indicating whether to output the state indices.

  • seed (Optional[int]) – a seed to be used as RNG initializer for reproducibility purposes.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Union[List[int], List[str]]

time_correlations(sequence1, sequence2=None, time_points=1)[source]

The method computes the time autocorrelations of a single observed sequence of states or the time cross-correlations of two observed sequences of states.

Notes:

  • If the Markov chain has multiple stationary distributions, then None is returned.

  • If a single time point is provided, then a float is returned.

  • The method can be accessed through the following aliases: tc.

Parameters:
Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Optional[Union [float, List[float]]]

time_relaxations(sequence, initial_distribution=None, time_points=1)[source]

The method computes the time relaxations of an observed sequence of states with respect to the given initial distribution of the states.

Notes:

  • If the Markov chain has multiple stationary distributions, then None is returned.

  • If a single time point is provided, then a float is returned.

  • The method can be accessed through the following aliases: tr.

Parameters:

  • sequence (Union[List[int], List[str]]) – the observed sequence of states.

  • initial_distribution (Optional[Union [ndarray, spmatrix]]) – the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).

  • time_points (Union[int, List[int]]) – the time point or a list of time points at which the computation is performed.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

Optional[Union [float, List[float]]]

to_bounded_chain(boundary_condition)[source]

The method returns a bounded Markov chain by adjusting the transition matrix of the original process using the specified boundary condition.

Notes:

  • The method can be accessed through the following aliases: to_bounded.

Parameters:

boundary_condition (Union[float, int, str])

  • a number representing the first probability of the semi-reflecting condition;

  • a string representing the boundary condition type (either absorbing or reflecting).

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

MarkovChain

to_canonical_form()[source]

The method returns the canonical form of the Markov chain.

Notes:

  • The method can be accessed through the following aliases: to_canonical.

Return Type:

MarkovChain

to_dictionary()[source]

The method returns a dictionary representing the Markov chain.

Return Type:

Dict[Tuple[str, str], float]

to_file(file_path)[source]

The method writes a Markov chain to the given file.

Only csv, json, txt and xml files are supported; data format is inferred from the file extension.
Parameters:

file_path (Union[str, Path]) – the location of the file in which the Markov chain must be written.

Raises:
to_graph()[source]

The method returns a directed graph representing the Markov chain.

Return Type:

DiGraph

to_lazy_chain(inertial_weights=0.5)[source]

The method returns a lazy Markov chain by adjusting the state inertia of the original process.

Notes:

  • The method can be accessed through the following aliases: to_lazy.

Parameters:

inertial_weights (Union[float, int, ndarray, spmatrix]) – the inertial weights to apply for the transformation.

Raises:

ValidationError – if any input argument is not compliant.

Return Type:

MarkovChain

to_matrix()[source]

The method returns the transition matrix of the Markov chain.

Return Type:

ndarray

to_subchain(states)[source]

The method returns a subchain containing all the given states plus all the states reachable from them.

Notes:

  • The method can be accessed through the following aliases: to_sub.

Parameters:

states (Union[int, str, List[int], List[str]]) – the states to include in the subchain.

Raises:
Return Type:

MarkovChain

transition_probability(state_target, state_origin)[source]

The method computes the probability of a given state, conditioned on the process being at a given state.

Parameters:
Raises:

ValidationError – if any input argument is not compliant.

Return Type:

float