torchtsetlin.functional¶
torchtsetlin.functional ¶
Stateless building blocks of Tsetlin machine learning, expressed as tensor operations.
Everything here works on any device (CPU / CUDA / MPS) and never touches autograd.
The model classes in :mod:torchtsetlin.models are thin, stateful wrappers around
these functions, so advanced users can compose their own Tsetlin machine variants.
Conventions¶
- A literal vector is
[x, ~x](length2FforFBoolean features). - Each clause owns one Tsetlin automaton (TA) per literal. Its state is an integer
in
[0, 2N-1]; states>= Nmean the literal is included (memorized) in the clause, states< Nmean excluded (forgotten).Nisn_states. - Clause output is the AND of its included literals. An empty clause (no included
literal) evaluates to
1during learning and0during prediction. - Feedback probabilities follow the specificity
s: memorize probability(s-1)/s, forget probability1/s(Type Ia/Ib), Type II is deterministic.
to_literals ¶
Turn Boolean features x of shape (..., F) into literals [x, ~x] of
shape (..., 2F) with the requested floating dtype (for matmul-based evaluation).
Source code in src/torchtsetlin/functional.py
clause_violations ¶
Number of included but False literals per clause.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
literals
|
Tensor
|
|
required |
include
|
Tensor
|
|
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
|
Source code in src/torchtsetlin/functional.py
clause_outputs ¶
clause_outputs(literals: Tensor, include: Tensor, include_count: Optional[Tensor] = None, empty_value: bool = True) -> Tensor
Evaluate all clauses on a batch of literal vectors.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
literals
|
Tensor
|
|
required |
include
|
Tensor
|
|
required |
include_count
|
Optional[Tensor]
|
optional |
None
|
empty_value
|
bool
|
value of a clause without included literals ( |
True
|
Returns:
| Type | Description |
|---|---|
Tensor
|
|
Source code in src/torchtsetlin/functional.py
vote_sums ¶
Weighted vote (class) sums clause_out @ weights optionally clamped to [-T, T].
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
clause_out
|
Tensor
|
|
required |
weights
|
Tensor
|
|
required |
T
|
Optional[float]
|
vote margin; if given the sums are clamped to |
None
|
Source code in src/torchtsetlin/functional.py
feedback_probabilities ¶
Probability of giving feedback to a clause given the (clamped) vote sum.
- For the target class the probability is
(T - v) / (2T): far below the margin -> update aggressively, at the margin -> stop. - For a negative class it is
(T + v) / (2T).
Source code in src/torchtsetlin/functional.py
sample_negative_classes ¶
sample_negative_classes(y: Tensor, n_classes: int, votes: Optional[Tensor] = None, focused: bool = False, generator: Optional[Generator] = None, T: Optional[float] = None) -> Tensor
Sample one negative (non-target) class per example.
With focused=True ("focused negative sampling") a non-target class k is sampled
with probability proportional to its negative-feedback probability
(T + clip(v_k, -T, T)) / 2T, as in tmu and green-tsetlin, so the classes the machine
confuses with the target are corrected first. This needs the vote margin T.
Otherwise sampling is uniform.
Source code in src/torchtsetlin/functional.py
type_i_counts ¶
type_i_counts(literals: Tensor, sel_fire: Tensor, sel_nofire: Tensor) -> Tuple[Tensor, Tensor, Tensor]
Aggregate Type I feedback events over a batch.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
literals
|
Tensor
|
|
required |
sel_fire
|
Tensor
|
|
required |
sel_nofire
|
Tensor
|
|
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
|
Tensor
|
|
Tensor
|
literal |
Tuple[Tensor, Tensor, Tensor]
|
(all literals are candidates for forgetting). |
Source code in src/torchtsetlin/functional.py
type_ii_counts ¶
Aggregate Type II feedback events: n2[j,k] counts (example, clause) pairs where the
clause matched the example, received Type II feedback and literal k was False.
Source code in src/torchtsetlin/functional.py
set_compile ¶
Enable or disable the torch.compile kernels of the update (default: enabled, or
the environment variable TORCHTSETLIN_COMPILE=0 to disable). The compiled and eager
paths draw from the same distributions but consume random numbers differently, so a seeded
run reproduces only under the same setting.
Source code in src/torchtsetlin/functional.py
count_included ¶
Included literals per clause, (state >= n_states).sum(dim=1), in row blocks.
The one-shot sum materialises a bool mask plus an int64 copy of it (9 bytes per automaton:
7.5 GiB at 640 000 clauses x 1 568 literals, which ran a 24 GB GPU out of memory). Blocks of
_COUNT_BLOCK_ELEMENTS automata bound that scratch to ~0.6 GiB at the same speed. The
block bounds depend only on the shape, so the loop is safe inside a captured CUDA graph.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state
|
Tensor
|
|
required |
n_states
|
int
|
|
required |
out
|
Optional[Tensor]
|
optional |
None
|
Returns:
| Type | Description |
|---|---|
Tensor
|
|
Source code in src/torchtsetlin/functional.py
apply_feedback ¶
apply_feedback(state: Tensor, n_true: Tensor, n_false: Tensor, n_ib: Tensor, n2: Tensor, *, n_states: int, s: float, boost_true_positive: bool = True, max_included_literals: Optional[int] = None, include_count: Optional[Tensor] = None, literal_active: Optional[Tensor] = None, generator: Optional[Generator] = None, budget_mode: str = 'soft') -> Tensor
Turn aggregated feedback counts into Tsetlin automata transitions (in place).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state
|
Tensor
|
|
required |
n_true
|
Tensor
|
|
required |
n_false
|
Tensor
|
|
required |
n_ib
|
Tensor
|
|
required |
n2
|
Tensor
|
|
required |
n_states
|
int
|
|
required |
s
|
float
|
specificity; memorize probability |
required |
boost_true_positive
|
bool
|
if |
True
|
max_included_literals
|
Optional[int]
|
clause size budget |
None
|
include_count
|
Optional[Tensor]
|
|
None
|
literal_active
|
Optional[Tensor]
|
optional |
None
|
generator
|
Optional[Generator]
|
optional RNG. |
None
|
budget_mode
|
str
|
|
'soft'
|
Returns:
| Type | Description |
|---|---|
Tensor
|
The updated |
Source code in src/torchtsetlin/functional.py
463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 | |
apply_feedback_blocks ¶
apply_feedback_blocks(state: Tensor, n_true: Tensor, n_false: Tensor, n_ib: Tensor, n2: Tensor, blocks: Sequence[Tuple[Tensor, float]], *, n_states: int, boost_true_positive: bool = True, max_included_literals: Optional[int] = None, include_count: Optional[Tensor] = None, literal_active: Optional[Tensor] = None, generator: Optional[Generator] = None) -> Tensor
:func:apply_feedback with a different specificity per block of literals.
blocks is a list of (columns, s): the literal columns (a long tensor into the
2F axis) that use specificity s. Every literal must be in exactly one block. The
clause budget is judged on the whole row (include_count before the update), so a
clause over budget gets Type Ib instead of Type Ia in every block, as with a scalar s.
Only the soft budget is supported. Used by the graph Tsetlin machine for a specificity
per layer (s=(s0, s1, ...), as the reference implementation allows).
The count tensors are read, not modified. Returns state, updated in place.
Source code in src/torchtsetlin/functional.py
sample_forget_events ¶
sample_forget_events(counts: Tensor, n_literals: int, p: float, generator: Optional[Generator] = None) -> Tuple[Tensor, Tensor]
Sparse exact draw of per-literal forgetting for row-broadcast event counts.
Row c holds counts[c] Bernoulli(p) trials for each of its n_literals
literals (Type Ib feedback applies the same count to every literal of a clause). Every
literal therefore needs a Binomial(counts[c], p) decrement. Instead of one draw per
literal, the successes of each row's counts[c] * n_literals trials are generated by
geometric gaps, so the work scales with the number of successes. Trial t of row c
belongs to literal t // counts[c].
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
counts
|
Tensor
|
|
required |
n_literals
|
int
|
literals per row |
required |
p
|
float
|
success probability, |
required |
generator
|
Optional[Generator]
|
optional RNG. |
None
|
Returns:
| Type | Description |
|---|---|
Tensor
|
|
Tensor
|
and the number of successes at each ( |
Source code in src/torchtsetlin/functional.py
apply_feedback_rows ¶
apply_feedback_rows(state: Tensor, n_true: Tensor, n_false: Tensor, n_ib: Tensor, n2: Tensor, rows: Tensor, *, n_states: int, s: float, boost_true_positive: bool = True, max_included_literals: Optional[int] = None, include_count: Optional[Tensor] = None, literal_active: Optional[Tensor] = None, generator: Optional[Generator] = None, budget_mode: str = 'soft') -> Optional[Tuple[Tensor, Tensor]]
:func:apply_feedback for count tensors that are zero outside rows.
Type Ia and Type II counts only exist for clauses that matched and were selected, a small
fraction of all clauses. Those rows are updated as a dense slab; the Type Ib forgetting of
every other clause is drawn sparsely with :func:sample_forget_events. The transitions
follow the same distribution as :func:apply_feedback. The count tensors are read, not
modified.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rows
|
Tensor
|
|
required |
Returns:
| Type | Description |
|---|---|
Optional[Tuple[Tensor, Tensor]]
|
|
Optional[Tuple[Tensor, Tensor]]
|
positions changed elsewhere, so that cached include masks can be refreshed |
Optional[Tuple[Tensor, Tensor]]
|
incrementally. |
Source code in src/torchtsetlin/functional.py
658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 | |
commit_single_example ¶
commit_single_example(state: Tensor, include: Tensor, include_count: Tensor, literals: Tensor, fire_i: Tensor, nofire_i: Tensor, fire_ii: Tensor, *, n_states: int, s: float, boost_true_positive: bool = True, max_included_literals: Optional[int] = None, literal_active: Optional[Tensor] = None, literals_ii: Optional[Tensor] = None) -> Optional[Tensor]
The feedback of one example, applied by a single fused torch.compile kernel.
Equivalent in distribution to accumulating the example's counts (the outer products of
fire_i / nofire_i / fire_ii with the literals) and calling
:func:apply_feedback with the soft budget, but without materialising any (C, 2F)
count tensor and without host synchronisation. Used by sequential mode on CUDA.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state
|
Tensor
|
|
required |
include
|
Tensor
|
|
required |
include_count
|
Tensor
|
|
required |
literals
|
Tensor
|
|
required |
fire_i
|
Tensor
|
|
required |
nofire_i
|
Tensor
|
|
required |
fire_ii
|
Tensor
|
|
required |
n_states
|
int
|
as in :func: |
required |
s
|
float
|
as in :func: |
required |
boost_true_positive
|
bool
|
as in :func: |
True
|
max_included_literals
|
Optional[int]
|
as in :func: |
None
|
literal_active
|
Optional[Tensor]
|
as in :func: |
None
|
literals_ii
|
Optional[Tensor]
|
literals of the Type II events, shaped like |
None
|
Returns:
| Type | Description |
|---|---|
Optional[Tensor]
|
The new |
Optional[Tensor]
|
(nothing has been modified; the caller falls back to :func: |
Source code in src/torchtsetlin/functional.py
commit_dense ¶
commit_dense(state: Tensor, include: Tensor, include_count: Tensor, n_true: Tensor, n_false: Tensor, n_ib: Tensor, n2: Tensor, *, n_states: int, s: float, boost_true_positive: bool = True, max_included_literals: Optional[int] = None, literal_active: Optional[Tensor] = None, max_events: Optional[int] = None) -> Optional[Tensor]
:func:apply_feedback (soft budget) as one fused torch.compile kernel that also
refreshes the include cache.
The binomials are drawn inside the kernel by inverse-CDF lookup, so the update reads the counts and the state once and writes the state, the include mask and zeroed counts once, instead of the ~20 full passes of the eager update. Used by batched updates on CUDA.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
state
|
Tensor
|
|
required |
include
|
Tensor
|
|
required |
include_count
|
Tensor
|
|
required |
n_true
|
Tensor
|
as in :func: |
required |
n_false
|
Tensor
|
as in :func: |
required |
n_ib
|
Tensor
|
as in :func: |
required |
n2
|
Tensor
|
as in :func: |
required |
n_states
|
int
|
as in :func: |
required |
s
|
float
|
as in :func: |
required |
boost_true_positive
|
bool
|
as in :func: |
True
|
max_included_literals
|
Optional[int]
|
as in :func: |
None
|
literal_active
|
Optional[Tensor]
|
as in :func: |
None
|
max_events
|
Optional[int]
|
an upper bound on any count (e.g. the examples in the update). With it the table width is chosen without reading the counts back to the host. |
None
|
Returns:
| Type | Description |
|---|---|
Optional[Tensor]
|
The new |
Optional[Tensor]
|
modified) when the kernel cannot be used: compilation disabled or failed, or more than |
Optional[Tensor]
|
|
Optional[Tensor]
|
eager path. |
Source code in src/torchtsetlin/functional.py
927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 | |
predict_proba_from_votes ¶
Softmax over vote sums normalised by T – a convenient (heuristic) probability.
votes / T lies in [-1, 1]; dividing by temperature sharpens (<1) or flattens
(>1) the distribution.
Source code in src/torchtsetlin/functional.py
confidence_from_votes ¶
Confidence in the predicted class, as in Chapter 7 of An Introduction to Tsetlin
Machines: the winning vote sum normalised to [0, 1] by the vote margin,
(v_max + T) / (2T). Returns shape (B,).
Source code in src/torchtsetlin/functional.py
boolean_pool ¶
Pool Boolean planes (B, Z, H, W) by an integer factor.
mode="or" keeps a bit that is set anywhere in the window (the Boolean analogue of
max-pooling, and what CTM-UNet uses between blocks); mode="and" keeps only bits set
everywhere in it. Both stay Boolean, so the result can be fed straight back into another
Tsetlin machine — no float parameter and no gradient anywhere, unlike a strided
convolution.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
x
|
Tensor
|
|
required |
factor
|
int
|
pooling window and stride. |
2
|
mode
|
str
|
|
'or'
|
Returns:
| Type | Description |
|---|---|
Tensor
|
|
Source code in src/torchtsetlin/functional.py
thermometer_cast ¶
Turn vote sums into Boolean planes without losing their ordering.
A Tsetlin machine can only read Boolean features, so passing one machine's votes to
another needs a Booleanization that preserves how strong the vote was. A thermometer
does: the planes of v are [v >= l] for each level l, so a larger vote sets a
superset of the bits a smaller one does, and a clause can refer to "at least this
confident" with one literal.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
votes
|
Tensor
|
|
required |
levels
|
Sequence[float]
|
ascending thresholds, e.g. |
required |
Returns:
| Type | Description |
|---|---|
Tensor
|
|
Tensor
|
vote sums. Channel order is class-major: all levels of class 0, then class 1, ... |