GlossaryGlossary

What Is a Permutation Test? Definition and Example

A simple way to ask “could luck explain this?” about a strategy or an analyst’s record, with a worked example.

Published Updated 2 min readHow we research

Key takeaways

  • A permutation test checks whether a result could be luck by shuffling the data many times and counting how often the shuffled versions do as well as the real one.
  • The share of shuffles that match or beat the real result is the p-value.
  • Base rates matter: in Exent’s 7-year study of 820,838 analyst calls with a price target (as of Nov 16, 2026), 53.0% of targets were hit, so a record a few points above that is not remarkable on its own.

Which stocks do Wall Street’s best‑ranked analysts agree on? See them in Exent Gems →

What is a permutation test?

It is a statistical test that needs no assumptions about bell curves. You break the link you care about, for example between a signal and the returns that followed, by shuffling one side at random. If the real result rarely appears in the shuffles, it is unlikely to be chance. It sits next to out-of-sample testing as a guard against fooling yourself.

Put this to work: see which stocks Wall Street’s best‑ranked analysts agree on in Exent Gems.

How does it work?

  1. Compute the statistic on the real data, such as hit rate or average return after a signal.
  2. Shuffle which outcome goes with which signal, and recompute. Repeat thousands of times.
  3. p-value = shuffles at least as good as the real result ÷ total shuffles

Worked example

Illustrative analyst testValue
Calls made60
Calls right38 (63%)
Shuffles run10,000
Shuffles with 38+ right420
p-value0.042

Illustrative only. A p-value of 0.042 says random assignment matched this record about 4 times in 100. Now the catch: among a thousand analysts with no skill, about 50 would pass a 5% test by chance alone. That is why analyst success rates need long records and comparison with the base rate.

Why it matters for a self‑directed investor

Track records are everywhere: funds, newsletters, analysts, strategies. A permutation test is the quickest way to ask whether one is better than a coin. See skill vs luck in returns and how accurate analysts are.

Common mistakes

  • Shuffling in a way that destroys real structure, such as market-wide moves that hit all stocks at once.
  • Testing hundreds of ideas and celebrating the few with low p-values.
  • Reading a low p-value as proof of a large or lasting edge.
  • Running too few shuffles to estimate small p-values.

Check analysts’ track records with Exent

  1. Stocks Radar ranks thousands of Wall Street analysts by past accuracy, with each analyst’s hit rate and number of calls.
  2. Gems flag stocks where several of the best‑rated analysts agree.

Exent is our product. Educational content, not investment advice.

Frequently asked questions

It estimates how often a result as good as the observed one would appear if the signal or the person had no real skill.

Thousands are typical. More shuffles give a more precise p-value, which matters when the p-value is small.

No. A backtest measures how a rule would have performed. A permutation test asks whether that performance stands out from what chance would produce.

Get the picks Wall Street’s best analysts agree on.

Exent tracks 4,000+ analysts, keeps only the elite few with proven records, and surfaces the Gems where they agree.

See this week’s GemsLimited offer$1.95 for full access· full access

Educational content, not investment advice. Examples are illustrative; past performance does not guarantee future results.

DISCLAIMER: Exent AI is an informational research platform and does not provide investment advice, financial advice, or act as an investment adviser, broker-dealer, or fiduciary. All content, analytics, and model outputs are provided for educational and informational purposes only and should not be relied upon as a recommendation to buy, sell, or hold any financial instrument. Investing involves risk, including the possible loss of principal. Exent AI does not execute trades, hold customer funds, or provide brokerage services. Past performance and hypothetical results are not indicative of future results. Third-party data and AI-generated outputs may be incomplete or inaccurate.