`psp.clobe`#

Global benchmarks for computational models: a PyTorch port of the R package clobe 0.8.0.

The tools used to compare model and human heterogeneity (Dome and Wills, 2025): inequality matrices, g-distance, and the counting and enumeration of the ordinal patterns (semiorders and weak orderings) that make up the universal set.

The port follows the conventions of the rest of psp rather than those of R:

A set of inequality matrices has shape (n, k, k), matrix first, as psp.PSPResult.ordinal_patterns does; the R package uses (k, k, n). A single (k, k) matrix is treated as a set of one.
Indices are 0-based.
imac() is batched, so it can be handed to psp.PSP() as discretize.
Counts are Python integers and therefore exact, however large.
brutes() runs on any torch device instead of a pool of threads.

One file per function, as in the R package:

psp.clobe.imac: imac(), the inequality matrix of a result.
psp.clobe.patterns: patterns(), the unique patterns in a set of results.
psp.clobe.gdistance: gdistance(), model against human heterogeneity.
psp.clobe.pattern_distance: pattern_distance(), Kemeny and Hamming distances.
psp.clobe.combinatorics: stirling2(), fubini() and semiorders(), the size of the universal set.
psp.clobe.orderings: orderings(), every semiorder or weak ordering.
psp.clobe.brutes: brutes(), the patterns a coarse measurement scale can produce.

Inequality matrices hold 1 (larger), -1 (smaller) or 0 (approximately equal) above the diagonal, comparing stimulus i with stimulus j > i, and 2 on and below the diagonal, which carry no information.

Dome, L., & Wills, A. J. (2025). g-Distance: On the comparison of model and human heterogeneity. Psychological Review, 132(3), 632-655. https://doi.org/10.1037/rev0000550

>>> import torch
>>> from psp import clobe
>>> clobe.imac(torch.tensor([0.9, 0.5, 0.45]), (0.1, -0.1))
tensor([[2., 1., 1.],
        [2., 2., 0.],
        [2., 2., 2.]])

`imac`(values, thresholds[, ordering, inclusive])	Inequality matrix constructor: the ordinal pattern of a result.
`patterns`(data, thresholds[, ordering, inclusive])	The unique patterns in results, one row per participant or model run.
`gdistance`(human, model, universal, weight, ...)	g-distance between the heterogeneity of humans and of a model (Dome and Wills, 2025).
`pattern_distance`(x[, y, method])	Distances between every pattern in `x` and every pattern in `y`.
`semiorders`(k)	The number of labelled semiorders of `k` items: the universal set of `imac()`.
`fubini`(k[, levels])	The number of weak orderings of `k` items (the Fubini or ordered Bell number).
`stirling2`(n, k)	Stirling number of the second kind: ways to split `n` items into `k` non-empty groups.
`orderings`(length[, ordering])	Every pattern of `length` stimuli: every semiorder, or every weak ordering.
`brutes`(dependent, length, thresholds[, ...])	The patterns a measure with the given values can produce, by brute force.

Inequality matrices#

psp.clobe.imac(values, thresholds, ordering='semiorder', *, inclusive=False)[source]#

Inequality matrix constructor: the ordinal pattern of a result.

Compares every pair of values. Above the diagonal, cell (i, j) is 1 if values[i] - values[j] > upper, -1 if it is < lower and 0 (approximately equal) otherwise; on and below the diagonal it is 2. With ordering="weak" only ties count as approximately equal, whatever the thresholds. A difference within TOLERANCE of a threshold counts as equal to it; for float32 and lower precision, within 8 eps of the dtype.

Batched over leading dimensions, so imac fits the discretize contract of psp.PSP(): (n, k) in, (n, k, k) out.

Parameters:

values: Results of shape (..., k), one value per stimulus (e.g. response probabilities).
thresholds: (upper, lower) with upper >= lower, e.g. (0.1, -0.1).
ordering: "semiorder" (the default) or "weak".
inclusive: Count a difference equal to a threshold as larger (>= upper) or smaller (<= lower), without a tolerance, instead of approximately equal. This is the rule of clobe before 0.8.0, used by psp.models.oscar. Semiorders only.

Returns:

torch.Tensor: Shape (..., k, k), the floating dtype of values (integers become the default dtype). A difference that is NaN stays NaN; the R package returns 0.

Examples

>>> imac(torch.tensor([[0.4, 0.3, 0.2], [0.2, 0.2, 0.9]]), (0.1, -0.1))
tensor([[[ 2.,  0.,  1.],
         [ 2.,  2.,  0.],
         [ 2.,  2.,  2.]],

        [[ 2.,  0., -1.],
         [ 2.,  2., -1.],
         [ 2.,  2.,  2.]]])

As the discretize of a run:

>>> from functools import partial
>>> discretize = partial(imac, thresholds=(0.1, -0.1))

A difference of exactly a threshold, by default and with inclusive:

>>> values = torch.tensor([0.5, 0.25], dtype=torch.float64)
>>> imac(values, (0.25, -0.25))[0, 1].item()
0.0
>>> imac(values, (0.25, -0.25), inclusive=True)[0, 1].item()
1.0

psp.clobe.patterns(data, thresholds, ordering='semiorder', *, inclusive=False)[source]#

The unique patterns in results, one row per participant or model run.

Each row is turned into an inequality matrix with imac(). The output is ready for gdistance(): each pattern appears once, with its frequency.

Parameters:

data: (rows, k) numbers, one column per stimulus: a tensor, an array or anything torch.as_tensor() accepts, such as DataFrame.to_numpy().
thresholds, ordering, inclusive: As in imac().

Examples

>>> data = torch.tensor([[0.9, 0.5, 0.1], [0.1, 0.5, 0.9], [0.8, 0.5, 0.2]])
>>> out = patterns(data, (0.1, -0.1))
>>> out.n, out.counts, out.pattern
(2, tensor([2, 1]), tensor([0, 1, 0]))

class psp.clobe.PatternsResult(inequality_matrices, counts, frequencies, pattern)[source]#

The unique patterns in a set of results, returned by patterns().

Attributes:

inequality_matrices: The unique patterns in order of first appearance, (n, k, k).
counts: How many rows of the data have each pattern, (n,) int64.
frequencies: counts as proportions of the rows, (n,) float64.
pattern: For each row of the data, the index of its pattern, (rows,) int64.

property n#: Number of unique patterns.

psp.clobe.TOLERANCE = 1.4901161193847656e-08#: Convert a string or number to a floating point number, if possible.

g-distance and distances between patterns#

psp.clobe.gdistance(human, model, universal, weight, frequencies)[source]#

g-distance between the heterogeneity of humans and of a model (Dome and Wills, 2025).

Two matrices are the same pattern if their strict upper triangles are equal; any numeric coding of the relations works, not only -1, 0 and 1.

Parameters:

human: The patterns people show, (n, k, k) or one (k, k) matrix, each pattern once (see patterns()).
model: The patterns the model produces, (m, k, k) or (k, k), e.g. psp.PSPResult.ordinal_patterns.
universal: The number of possible patterns, e.g. semiorders(k) or brutes(...).n.
weight: Weight of accommodation, between 0 and 1; 1 - weight is the weight of excess flexibility.
frequencies: The proportion of people with each human pattern, (n,).

Warns:

UserWarning: If frequencies do not sum to 1 or human repeats a pattern, in which case alpha is not a proportion; or if universal equals the number of human patterns, in which case beta is NaN and left out of the distance.

Examples

>>> from psp.clobe import patterns, semiorders
>>> human = patterns(torch.tensor([[0.9, 0.5, 0.1], [0.8, 0.5, 0.2], [0.1, 0.5, 0.9]]),
...                  (0.1, -0.1))
>>> model = patterns(torch.tensor([[0.9, 0.5, 0.1], [0.5, 0.5, 0.5]]), (0.1, -0.1))
>>> out = gdistance(human.inequality_matrices, model.inequality_matrices,
...                 universal=semiorders(3), weight=0.5, frequencies=human.frequencies)
>>> round(out.gdistance, 4), round(out.alpha, 4), round(out.beta, 4), out.accommodation
(0.2393, 0.6667, 0.0588, tensor([0]))

class psp.clobe.GDistanceResult(gdistance, alpha, beta, accommodation)[source]#

The result of gdistance().

Attributes:

gdistance: sqrt(weight * (1 - alpha)^2 + (1 - weight) * beta^2); without beta when it is NaN.
alpha: Accommodation: the summed frequency of the human patterns the model produces.
beta: Excess flexibility: the unique model patterns that no human shows, as a proportion of the patterns in the universal set that no human shows. NaN when humans show every pattern of the universal set.
accommodation: Indices (0-based) of the human patterns the model produces, (m,) int64.

psp.clobe.pattern_distance(x, y=None, method='kemeny')[source]#

Distances between every pattern in x and every pattern in y.

Compares the strict upper triangles. "kemeny" sums the absolute differences of the cells, so reversing a relation (1 to -1) counts 2 and breaking a tie counts 1; "hamming" counts the cells that differ.

Parameters:

x, y: (n, k, k) and (m, k, k) patterns, or single (k, k) matrices; y defaults to x.
method: "kemeny" (the default) or "hamming".

Returns:

torch.Tensor: (n, m) float64.

Examples

>>> from psp.clobe import imac
>>> a = imac(torch.tensor([0.9, 0.5, 0.1]), (0.1, -0.1))  # 1 > 2 > 3
>>> c = imac(torch.tensor([0.1, 0.5, 0.9]), (0.1, -0.1))  # 3 > 2 > 1
>>> pattern_distance(a, c), pattern_distance(a, c, method="hamming")
(tensor([[6.]], dtype=torch.float64), tensor([[3.]], dtype=torch.float64))

The universal set#

psp.clobe.semiorders(k)[source]#

The number of labelled semiorders of k items: the universal set of imac().

The exponential generating function is C(1 - exp(-x)), where C is that of the Catalan numbers, which gives an alternating sum over Stirling numbers.

>>> [semiorders(k) for k in range(7)]
[1, 1, 3, 19, 183, 2371, 38703]

psp.clobe.fubini(k, levels=None)[source]#

The number of weak orderings of k items (the Fubini or ordered Bell number).

Parameters:

k: Number of items (stimuli).
levels: At most this many tied groups, e.g. the number of distinct values a measure can take; None for no limit.

Examples

>>> [fubini(k) for k in range(6)], fubini(4, levels=3)
([1, 1, 3, 13, 75, 541], 51)

psp.clobe.stirling2(n, k)[source]#

Stirling number of the second kind: ways to split n items into k non-empty groups.

>>> stirling2(5, 2), stirling2(10, 3), stirling2(3, 5)
(15, 9330, 0)

psp.clobe.orderings(length, ordering='semiorder')[source]#

Every pattern of length stimuli: every semiorder, or every weak ordering.

The patterns any imac() result can take, so orderings(k).n equals semiorders(k) and orderings(k, "weak").n equals fubini(k).

Examples

>>> orderings(2).inequality_matrices
tensor([[[ 2, -1],
         [ 2,  2]],

        [[ 2,  0],
         [ 2,  2]],

        [[ 2,  1],
         [ 2,  2]]], dtype=torch.int8)

class psp.clobe.OrderingsResult(inequality_matrices)[source]#

The result of orderings().

Attributes:

inequality_matrices: Every pattern, (n, k, k) int8, in the order of the R package.

property n#: Number of patterns.

psp.clobe.brutes(dependent, length, thresholds, ordering='semiorder', *, return_matrices=True, return_permutations=False, inclusive=False, device=None, chunk_size=4194304)[source]#

The patterns a measure with the given values can produce, by brute force.

Assigns the m values of dependent to length stimuli in every one of the m^length ways, with repetition, and collects the unique patterns imac() gives them. This is the universal set when the measurement scale is too coarse to realise every semiorder (or weak ordering).

Parameters:

dependent: The values the measure can take, e.g. torch.linspace(0, 1, 11). Compared in float64, with the tolerance of their own dtype (see imac()).
length: Number of stimuli.
thresholds, ordering, inclusive: As in imac().
return_matrices: Return the patterns; otherwise only their number.
return_permutations: Also return every assignment of values to stimuli.
device: Where to do the work, e.g. "cuda"; the results are returned on the CPU.
chunk_size: Assignments evaluated at once; lower it if the device runs out of memory.

Examples

>>> [brutes(torch.linspace(0, 1, 11), k, (0.1, -0.1)).n for k in range(1, 5)]
[1, 3, 19, 183]
>>> brutes(torch.tensor([0.0, 1.0, 2.0]), 4, (0.5, -0.5), "weak").n  # at most 3 levels
51

class psp.clobe.BrutesResult(n, inequality_matrices, permutations)[source]#

The result of brutes().

Attributes:

n: Number of unique patterns.
inequality_matrices: The unique patterns, (n, k, k) int8, or None if not requested.
permutations: Every assignment of the dependent values to the stimuli, (m^k, k) float64, the last stimulus changing fastest, or None if not requested.

psp.clobe#

Inequality matrices#

g-distance and distances between patterns#

The universal set#

`psp.clobe`#