Choosing between options that differ on several criteria at once is one of the most common decisions there is: suppliers, laptops, job offers, research proposals. Multi-criteria decision analysis (MCDA) gives that choice a method. You score every option on every criterion, weigh the criteria, and let an algorithm such as TOPSIS rank the options.
The ranking comes out looking exact, down to four decimals. This tutorial uses mcdakit, my open-source Python library, to ask two questions that most tools never ask:
- Would the ranking change if a bad option were removed?
- How far would the weights have to move before a different option won?
Install
mcdakit needs Python 3.9 or later, and numpy is its only dependency.
pip install git+https://github.com/aitmlouk/mcdakit
The decision
We are choosing between four suppliers on four criteria. Price and lead time are costs, so lower is better. Quality and support are benefits, so higher is better. Supplier D is the cheapest, but it is poor on everything else.
from mcdakit import Criterion, rank, reversal_check, sensitivity
criteria = [
Criterion("Price", weight=0.40, direction="cost", bounds=(2.00, 4.00)),
Criterion("Quality", weight=0.25, direction="benefit", bounds=(0, 10)),
Criterion("Lead time", weight=0.20, direction="cost", bounds=(5, 35)),
Criterion("Support", weight=0.15, direction="benefit", bounds=(0, 10)),
]
matrix = [
[2.75, 7.0, 14, 8.0], # Supplier A
[2.90, 8.5, 16, 8.0], # Supplier B
[3.40, 9.0, 11, 7.0], # Supplier C
[2.20, 3.0, 32, 2.0], # Supplier D: cheapest, but poor on everything else
]
labels = ["Supplier A", "Supplier B", "Supplier C", "Supplier D"]
bounds gives each criterion’s plausible range, fixed in advance. We will see in a moment why that matters.
Step 1: rank with TOPSIS
result = rank(matrix, criteria, method="topsis", labels=labels)
print(result)
topsis ranking:
1. Supplier B 0.7159
2. Supplier A 0.7044
3. Supplier C 0.6238
4. Supplier D 0.3666
Supplier B wins narrowly. Nobody would choose Supplier D.
Step 2: check for rank reversal
Suppose Supplier D drops out. It was never going to win, so the order of the others should not change. Let’s check:
check = reversal_check(matrix, criteria, method="topsis", labels=labels)
print(check["reversed"]) # True
print(check["cases"][0]["after"]) # ['Supplier A', 'Supplier C', 'Supplier B']
Removing the last-placed supplier sends the winner, Supplier B, from first to last. This is called rank reversal. It is not a bug in mcdakit or in your data. TOPSIS measures each option against the best and worst values found among the options themselves, so changing the set of options moves those reference points. Supplier D had the lowest price and the worst quality, lead time and support, so it set most of the reference points, and they were propping up Supplier B.
Step 3: rank against fixed bounds with SPOTIS
SPOTIS measures every option against a fixed ideal point built from the bounds, never against the other options. Adding or removing an option cannot move anyone else’s score.
result = rank(matrix, criteria, method="spotis", labels=labels)
print(result.winner) # Supplier A
print(result.reversal_free) # True
check = reversal_check(matrix, criteria, method="spotis", labels=labels)
print(check["reversed"]) # False
If you leave out bounds, mcdakit falls back to the observed minimum and maximum, warns you, and sets reversal_free to False. It will not quietly hand back a ranking that only looks safe.
Step 4: how fragile is the winner?
The weights (0.40, 0.25, 0.20, 0.15) were somebody’s judgement. A colleague might reasonably pick different ones. sensitivity finds, for each criterion, the smallest change to its weight that makes a different option win.
report = sensitivity(result)
print(report["level"], report["weakest"]) # fragile Quality
for row in report["rows"]:
if row["tolerance"] is None:
print(f"{row['name']:<10} never flips")
else:
print(f"{row['name']:<10} flips if it moves {row['tolerance']:.1%} "
f"{row['direction']} -> {row['flips_to']}")
Price flips if it moves 7.8% down -> Supplier B
Quality flips if it moves 3.9% up -> Supplier B
Lead time flips if it moves 8.8% down -> Supplier B
Support never flips
Each percentage is a share of the total weight. Raising the weight on quality from 0.25 to about 0.29 is enough to hand the win to Supplier B. mcdakit labels this decision fragile, its term for any winner that flips with a change of less than 10%.
That is the real finding. “A and B are practically tied, and the choice comes down to how much you value quality” is more honest, and more useful to a decision maker, than “A scored −0.3150”.
What to take away
- Always check a ranking for reversal before you trust it. With mcdakit, that is one call to
reversal_check. - If the set of options can change, use a reversal-free method such as SPOTIS and give it real bounds.
- Report how robust the winner is, not just who won. A narrow win is still a result, as long as you say it is narrow.
mcdakit has 17 ranking methods, including TOPSIS, VIKOR, PROMETHEE, ELECTRE and AHP, as well as weighting methods and group decisions. The documentation covers them all, and the source is on GitHub.