League Ratings

The following is an attempt to attach a numerical value ot the relative strength of the leagues in the pyramid, based on their relative performances in the FA Cup, FA Trophy and FA Vase in the 2003-4 season. The first table shows each league's rating, along with its level, and the number of matches used to calculate that rating.
The rating of each league can be used to work out the chance of a team from that league beating a team from another league. This works as follows:

Subtract the higher rating from the lower.
Look up this number in the upper row of the second hand table (or the nearest number to it)
The percentage given in the second lower row is the liklihood of the team from the stronger league beating the team form the weaker league.
Subtract this number from 100% to get the chance of the weaker league's team beating the stronger league's team.

The rather complicated details behind how these ratings are calculated are given after the two tables below. It is also woth noting that the rating of a league with fewer than 10 matches is almost always the result of the performances of only one or teams form that league, and so the ratings of such leagues should not be considered to be precise.
Key: P - Matches played; R - Rating.

League Level
to 03/04 2000/1 2001/2 * 2003/4 2004/5
P R P R P R Level P R
Premiership 1 63 100 56 100 48 100 1 53 100
Division 1 / FLC 2 53 94.47 41 90.79 46 92.90 2 39 90.98
Division 2 / League 1 3 72 93.84 57 85.57 54 92.27 3 56 85.11
Division 3 / League 2 4 55 91.18 51 87.84 50 93.10 4 46 85.32
Conference 5 130 89.64 77 81.71 96 90.54 5 90 81.84
Conference N 6 107 78.16
Conference S 6 104 78.65
Isthmian Prem 6 126 85.62 60 79.02 111 86.14 7 108 79.04
NPL Prem 6 127 84.61 54 75.03 99 85.63 7 82 73.49
Southern Prem 6 116 85.85 53 75.09 91 83.24 7 95 76.47
Isthmian 1 7 114 80.99 38 70.88 8 90 74.29
Isthmian 1N 7 106 79.53
Isthmian 1S 7 97 82.43
NPL 1 7 111 83.59 34 69.57 89 83.30 8 95 75.39
Southern East 7 103 81.51 38 72.31 94 80.39 8 85 73.65
Southern West 7 112 84.96 37 69.71 93 82.20 8 82 72.39
Comb Co Prem 8 65 75.78 32 70.52 68 73.36 9 56 65.14
Eastern Prem 8 112 80.40 57 74.54 85 77.01 9 81 72.01
Essex Senior 8 27 73.08 15 65.76 40 79.83 9 44 66.80
Hellenic Prem 8 54 78.96 30 67.85 62 79.29 9 66 69.11
Isthmian 2 8 119 80.30 56 74.26 60 80.78 9 65 71.18
Kent 8 64 76.14 24 67.82 61 78.04 9 68 72.06
Midland Alliance 8 111 79.61 43 68.52 71 78.60 9 93 71.31
Northern 1 8 127 85.13 63 77.22 90 80.45 9 92 69.58
Northern Co E Prem 8 95 81.96 45 72.43 82 77.16 9 79 73.24
NW Counties 1 8 110 80.72 54 74.97 105 76.09 9 95 69.58
Spartan SM P 8 74 75.54 39 71.84 58 81.11 9 61 69.28
Sussex 1 8 75 74.82 41 71.06 50 75.80 9 57 65.04
United Counites P 8 79 75.50 44 71.94 73 75.77 9 74 68.44
Wessex 1 8 89 77.30 46 70.81 98 76.34 9 89 72.28
Western P 8 84 81.05 44 70.91 72 73.16 9 87 73.56
Dorset 9 1 76.53
Eastern 1 9 19 63.73 32 66.09 10 52 67.97
Hampshire League 9 4 66.66 25 83.92
Hellenic East 9 2 59.85 16 78.16
Hellenic West 9 1 61.84 4 81.79 10 2 57.80
Isthmian 3 9 103 78.92 52 70.03
Kent County 9 4 79.99
Leicester Senior 9 18 67.44 3 77.04 10 20 63.64
Northern 2 9 39 77.01 23 66.45 54 76.36 10 55 62.87
Northern Counties East 1 9 53 79.48 28 68.92 62 77.21 10 48 64.96
NW Counties 2 9 43 79.00 23 72.68 44 70.40 10 50 64.39
Spartan SM 1 9 14 67.97 6 74.17 10 7 60.04
Sussex 2 9 8 68.29 8 67.07 26 71.61 10 25 63.21
United Counties 1 9 5 60.77 6 67.41 10 5 59.02
Wessex 2 9 10 13 65.71
West Midlands 9 2 61.94 18 79.06 10 23 66.76
Western 1 9 22 71.91 19 65.36 48 69.83 10 39 66.49
Central Midlands S 10 17 68.83 8 72.80 11 16 63.71
Devon 10 1 60.81 5 72.14 11 5 64.98
Manchester 10 1 68.08
Midland Combo 10 17 64.66 4 72.52 11 22 64.77
Midland League P 10 21 67.58
Northern Alliance 10 6 72.22 11 4 65.27
South West 10 5 79.11 10 73.97 23 78.08 11 28 69.01
Sussex 3 10 11 1 55.04
Wearside 10 1 58.92 5 73.15 11 1 59.58
West Cheshire 10 3 69.52 20 77.41 11 1 54.39
Central Midlands P/1 10 1 77.75 12 4 63.10
London Intermediate 10 5 83.81
Notts Alliance 12 3 68.52

* The results for the 2001/2 season do not include the qualifying rounds for the FA Cup, though I hope to be able to include thse soon.

League	Level to 03/04	2000/1	2001/2 *	2003/4		2004/5
P	R	P	R	P	R		Level	P	R
Premiership	1	63	100	56	100	48	100		1	53	100
Division 1 / FLC	2	53	94.47	41	90.79	46	92.90		2	39	90.98
Division 2 / League 1	3	72	93.84	57	85.57	54	92.27		3	56	85.11
Division 3 / League 2	4	55	91.18	51	87.84	50	93.10		4	46	85.32
Conference	5	130	89.64	77	81.71	96	90.54		5	90	81.84
Conference N									6	107	78.16
Conference S									6	104	78.65
Isthmian Prem	6	126	85.62	60	79.02	111	86.14		7	108	79.04
NPL Prem	6	127	84.61	54	75.03	99	85.63		7	82	73.49
Southern Prem	6	116	85.85	53	75.09	91	83.24		7	95	76.47
Isthmian 1	7	114	80.99	38	70.88				8	90	74.29
Isthmian 1N	7					106	79.53
Isthmian 1S	7					97	82.43
NPL 1	7	111	83.59	34	69.57	89	83.30		8	95	75.39
Southern East	7	103	81.51	38	72.31	94	80.39		8	85	73.65
Southern West	7	112	84.96	37	69.71	93	82.20		8	82	72.39
Comb Co Prem	8	65	75.78	32	70.52	68	73.36		9	56	65.14
Eastern Prem	8	112	80.40	57	74.54	85	77.01		9	81	72.01
Essex Senior	8	27	73.08	15	65.76	40	79.83		9	44	66.80
Hellenic Prem	8	54	78.96	30	67.85	62	79.29		9	66	69.11
Isthmian 2	8	119	80.30	56	74.26	60	80.78		9	65	71.18
Kent	8	64	76.14	24	67.82	61	78.04		9	68	72.06
Midland Alliance	8	111	79.61	43	68.52	71	78.60		9	93	71.31
Northern 1	8	127	85.13	63	77.22	90	80.45		9	92	69.58
Northern Co E Prem	8	95	81.96	45	72.43	82	77.16		9	79	73.24
NW Counties 1	8	110	80.72	54	74.97	105	76.09		9	95	69.58
Spartan SM P	8	74	75.54	39	71.84	58	81.11		9	61	69.28
Sussex 1	8	75	74.82	41	71.06	50	75.80		9	57	65.04
United Counites P	8	79	75.50	44	71.94	73	75.77		9	74	68.44
Wessex 1	8	89	77.30	46	70.81	98	76.34		9	89	72.28
Western P	8	84	81.05	44	70.91	72	73.16		9	87	73.56
Dorset	9					1	76.53
Eastern 1	9			19	63.73	32	66.09		10	52	67.97
Hampshire League	9			4	66.66	25	83.92
Hellenic East	9			2	59.85	16	78.16
Hellenic West	9			1	61.84	4	81.79		10	2	57.80
Isthmian 3	9	103	78.92	52	70.03
Kent County	9	4	79.99
Leicester Senior	9			18	67.44	3	77.04		10	20	63.64
Northern 2	9	39	77.01	23	66.45	54	76.36		10	55	62.87
Northern Counties East 1	9	53	79.48	28	68.92	62	77.21		10	48	64.96
NW Counties 2	9	43	79.00	23	72.68	44	70.40		10	50	64.39
Spartan SM 1	9			14	67.97	6	74.17		10	7	60.04
Sussex 2	9	8	68.29	8	67.07	26	71.61		10	25	63.21
United Counties 1	9			5	60.77	6	67.41		10	5	59.02
Wessex 2	9								10	13	65.71
West Midlands	9			2	61.94	18	79.06		10	23	66.76
Western 1	9	22	71.91	19	65.36	48	69.83		10	39	66.49
Central Midlands S	10			17	68.83	8	72.80		11	16	63.71
Devon	10			1	60.81	5	72.14		11	5	64.98
Manchester	10					1	68.08
Midland Combo	10			17	64.66	4	72.52		11	22	64.77
Midland League P	10			21	67.58
Northern Alliance	10					6	72.22		11	4	65.27
South West	10	5	79.11	10	73.97	23	78.08		11	28	69.01
Sussex 3	10								11	1	55.04
Wearside	10			1	58.92	5	73.15		11	1	59.58
West Cheshire	10			3	69.52	20	77.41		11	1	54.39
Central Midlands P/1	10					1	77.75		12	4	63.10
London Intermediate	10	5	83.81
Notts Alliance	12			3	68.52

Difference 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 6 7 8
% Win 50% 52% 54% 56% 58% 60% 62% 64% 66% 67% 69% 73% 76% 79%
Difference 9 10 11 12 13 14 15 16 17 18 19 20-21 22-24 over 25
% Win 82% 84% 86% 88% 90% 92% 93% 95% 96% 96% 97% 98% 99% ~ 100%

Difference	0	0.5	1	1.5	2	2.5	3	3.5	4	4.5	5	6	7	8
% Win	50%	52%	54%	56%	58%	60%	62%	64%	66%	67%	69%	73%	76%	79%
Difference	9	10	11	12	13	14	15	16	17	18	19	20-21	22-24	over 25
% Win	82%	84%	86%	88%	90%	92%	93%	95%	96%	96%	97%	98%	99%	~ 100%

How the ratings are calculated

The results for the FA Cup, FA Trophy and FA Vase for each season were used to how the ratio of wins and loses between teams from any given pair of leagues. It was then assumed that the space of all matches follows a normal distribution with mean zero, and a given standard deviation (see below). The number of standard deviations away from the mean that the win-loss ratio reprsented was calculated. This was then declared to be the difference in rating between those two leagues, so the rating of a league with respect another league is known. The Premiership was given a rating of 100, and each league's rating is the average of its rating with respect to all other leagues. This leads to an iterative process, which fortunately converges.

There are three arbituary parameters which go into these ratings. The first is the rating of the Premiership. Changing this causes all other ratings to change by exactly the same amount, so the difference between the ratings are unchanged, and so probabilities of a team from one league beating a team from another is unchanged.
The second is the standard devitaion. It works out that when the probabilites are calculated from the differences in ratings, they remain unchanged. (You need the standard deviation to calculate the differences / probabiliites table)
The third relates to what happens when a league has perfect win (or loss) record against another league. According to the process described above, the ratings should be infinitely different. This is obviously impractical, and so a certain number of standard deviations is 'awarded' for a prefect win record (or subtracted for perfect loss record). This number was chosen to be equvilant to winning 95% of the time.