Primality Testing

Saravanan Vijayakumaran

Department of Electrical Engineering, IIT Bombay

October 12, 2025

Primality Testing

  • First efficient algorithms were developed in the 70s

  • Probabilistic algorithms with following guarantees

    • If input p is a prime, the algorithms always output “prime”
    • If p is composite, the algorithms might output “prime” with a negligible probability
  • A deterministic polynomial-time algorithm was found in 2002 by Agrawal, Kayal, Saxena (AKS)

    • Not used in practice due to slower running time

Modular Exponentiation

  • Computing a^b \bmod N for a \in \mathbb{Z}_N and exponent b can be done in \mathcal{O}(\| b \| \times \|N\|^2) time
  • The square and multiply algorithm \begin{align*} & x \leftarrow a, t \leftarrow 1 \\ & \textbf{while } b > 0\\ & \quad \textbf{if } b \text{ is odd}\\ &\quad \quad t \leftarrow t \cdot x \bmod N, \quad b \leftarrow b-1\\ & \quad \textbf{end if }\\ & \quad x \leftarrow x^2 \bmod N, \quad b \leftarrow b/2\\ & \textbf{return } t \end{align*}

A Primality Test Based on FLT

  • Fermat’s little theorem: If p is a prime and a is any integer not divisible by p, then a^{p-1} = 1 \bmod p.

  • The Primality Test

    • Choose a random integer a from \{1,2,3,\ldots,N-1\}
    • Compute a^{N-1} \bmod N
    • If a^{N-1} \neq 1 \bmod N, then N is not a prime
  • If a^{N-1} = 1 \bmod N, we get no information about the primality of N

Everything in \left(\mathbb{Z}_N^* \right)^c Proves that N is Composite

  • Suppose a \in \{1,2,\ldots,N-1\} and a \notin \mathbb{Z}_N^*

    • \gcd(a, N) \neq 1 implies \gcd(a^{N-1}, N) \neq 1
    • Then a^{N-1} \neq 1 \bmod N
    • Such an a proves that N is composite

\left(\mathbb{Z}_N^* \right)^c is a Small Set

  • Integers in \{1,\ldots,N-1\} not belonging to \mathbb{Z}_N^* are rare

  • If N is prime, then there are no such integers

  • For composite N = p_1^{e_1} \cdots p_k^{e_k}, we have |\mathbb{Z}_N^*| =\phi(N) =\prod_i p_i^{e_i-1}(p_i-1)

  • If p_1,p_2,\ldots,p_k are large primes, then the fraction of \{1,2,\ldots,N-1\} in \mathbb{Z}_N^* is close to 1

Witnesses for Compositeness in \mathbb{Z}_N^*

  • We say that an integer a \in \mathbb{Z}_N^* is a witness for compositeness of N if a^{N-1} \neq 1 \bmod N

  • \gcd(a, N) = 1 \implies \gcd(a^{N-1}, N) = 1

    • Xa^{N-1} + YN =1 for some integers X,Y
    • So Xa^{N-1} = 1 \bmod N
    • But a^{N-1} \bmod N may or may not be equal to 1
  • So the a’s in \mathbb{Z}_N^* may or may not be witnesses

Testing Primality When At Least One Witness Exists

  • Theorem If there exists a witness in \mathbb{Z}_N^* that N is composite, then at least half the elements of \mathbb{Z}_N^* are witnesses that N is composite

  • If a witness exists, the following primality test fails to detect the compositeness of N with probability at most 2^{-t}

    1. For i=1,2,\ldots,t, repeat steps 2 and 3.
    2. Pick a uniformly from \{1,2,\ldots,N-1\}.
    3. If a^{N-1} \neq 1 \bmod N, return “composite’”
    4. If all t iterations have a^{N-1} = 1 \bmod N, return “prime”

Number of Witnesses When At Least One Witness Exists

  • Theorem: Suppose there exists a witness that N is composite. Then at least half the elements of \mathbb{Z}_N^* are witnesses that N is composite.

  • Proof

    • Let \textsf{Bad} be the subset of \mathbb{Z}_N^* that are not witnesses
    • \textsf{Bad} is a subgroup of \mathbb{Z}_N^*
    • By the hypothesis, it is a strict subgroup of \mathbb{Z}_N^*
    • By Lagrange’s theorem, |\textsf{Bad}| \le |\mathbb{Z}_N^*|/2
    • So |\textsf{Bad}^c| \ge |\mathbb{Z}_N^*|/2

Carmichael Numbers

  • Composite numbers for which a^{N-1} = 1 \bmod N for all integers a \in \mathbb{Z}_N^*
  • There are infinitely many Carmichael numbers
  • Example: 561= 3 \cdot 11 \cdot 17
  • We cannot rely on the existence of at least one witness in constructing our primality test

Square Roots of Unity Modulo a Prime

  • We say that x \in \mathbb{Z}_N^* is a square root of 1 modulo N if x^2 = 1 \bmod N.

  • Lemma: If N is an odd prime, then the only square roots of 1 modulo N are \pm 1 \bmod N.

The Sequence of Square Roots of Unity Modulo an Odd Prime

  • For an odd prime N, we have N-1 = 2^r u where r \ge 1 and u is odd

  • Suppose a^{N-1} = a^{2^ru} = 1 \bmod N

  • Consider the sequence a^u \bmod N,a^{2u} \bmod N, \ldots, a^{2^r u} \bmod N

  • It can only be in one of the following forms

    • 1,1,1,1,\ldots,1
    • -1,1,1,\ldots,1
    • \star,\star,\ldots,\star,-1,1,1,\ldots,1
  • For an odd integer N, if we do not observe this sequence then N is composite

The Sequence of Square Roots of Unity Modulo an Odd Prime

  • In other words, when N is an odd prime, the sequence a^u \bmod N,a^{2u} \bmod N, \ldots, a^{2^r u} \bmod N satisfies the following conditions

    • Either a^u = 1 \bmod N
    • Or one of a^u \bmod N, a^{2u} \bmod N,\ a^{2^2 u} \bmod N,\ \dots,\ a^{2^{r-1} u} \bmod N is equal to -1.

Strong Witnesses

  • We say that a \in \mathbb{Z}_N^* is a strong witness that N is composite if

    • a^u \neq 1 \bmod N and

    • a^{2^i u} \neq -1 \bmod N for all i \in \{0,1,2,\ldots,r-1\}.

  • If we can find even one strong witness, we can conclude that N is composite.

  • Theorem Let N be an odd number that is not a prime power. Then at least half the elements of \mathbb{Z}_N^* are strong witnesses that N is composite.

    • A prime power is of the form p^r where p is a prime and r \ge 1

Strategy for Proving Strong Witness Count

  • Let \textsf{Bad} \subseteq \mathbb{Z}_N^* be the set of elements that are not strong witnesses

  • We can define a set \textsf{Bad}' such that

    • \textsf{Bad} \subseteq \textsf{Bad}'
    • \textsf{Bad}' is a strict subgroup of \mathbb{Z}_N^*
    • By Lagrange’s theorem, |\textsf{Bad}| \le |\textsf{Bad}'| \le |\mathbb{Z}_N^*|/2
    • Hence, at least half of \mathbb{Z}_N^* are strong witnesses

Defining \textsf{Bad}'

  • For N-1 = 2^r u, let i \in \{0,1,\ldots,r-1\} be the largest integer for which there exists an a \in \textsf{Bad} with a^{2^iu} = -1 \bmod N.
  • Since (-1)^u = -1 \bmod N, i is well-defined
  • Let \textsf{Bad}' = \{ a \in \mathbb{Z}_N^* \mid a^{2^iu} = \pm 1 \bmod N \}
  • \textsf{Bad}' is a subgroup of \mathbb{Z}_N^*

\textsf{Bad}' is a Strict Subgroup

  • \textsf{Bad}' = \{ a \in \mathbb{Z}_N^* \mid a^{2^iu} = \pm 1 \bmod N \}
  • Since N is an odd composite integer that is not a prime power, we have N = N_1 N_2 \text{ for } N_1, N_2 > 1 \text{ and } \gcd(N_1, N_2) = 1
  • By CRT, \mathbb{Z}_N^* \simeq \mathbb{Z}_{N_1}^* \times \mathbb{Z}_{N_2}^*
  • Let a \in \textsf{Bad}' be such that a^{2^i u} = -1 \bmod N
  • Let a \leftrightarrow (a_1, a_2), then a_1^{2^i u} = -1 \bmod N_1 \text{ and } a_2^{2^i u} = -1 \bmod N_2
  • Let b \in \mathbb{Z}_N^* such that b \leftrightarrow (a_1, 1). Then b^{2^i u} \leftrightarrow (a_1, 1)^{2^iu} = (-1,1) \not\leftrightarrow \pm 1
  • So b \not\in \textsf{Bad}' \implies \textsf{Bad}' \subsetneq \mathbb{Z}_N^*

Miller-Rabin Primality Test

  • Input: Odd integer N > 1 and parameter 1^t

  • Output: A decision as to whether N is prime or composite \begin{align*} & \text{{Compute $r \ge 1$ and odd $u$ such that $N-1 = 2^ru$}}\\ & \textbf{if } \text{$N$ is a perfect power, \textbf{return} ``composite''}\\ & \textbf{for } j=1 \text{ to } t:\\ &\quad a \leftarrow \{1,\ldots,N-1\}\\ &\quad \textbf{if } \text{$a^u \neq \pm 1 \bmod N$ and}\\ &\quad \quad \text{$a^{2^iu} \neq -1 \bmod N$ for $i \in \{1,\ldots,r-1\}$ } \\ &\quad \quad \text{\textbf{return} ``composite''}\\ &\quad \textbf{end if } \\ &\textbf{end for } \\ & \textbf{return } \text{``prime''} \end{align*}

Further reading

  • Section 9.2.2 of Katz and Lindell