| Home > | Liberal Scholarship > (( Essay )) |
Jesus Seminar Voting Algorithmby Lee Adams Young In every Red-vs-Pink decision, the algorithm used by the Jesus Seminar has the inadvertent effect of giving each Red vote three times the weight of each Pink vote. Also, in every Gray-vs-Black decision the Seminar's algorithm unintentionally gives each Black vote three times the weight of each Gray vote. The algorithm is unbiased for Pink-vs-Gray decisions. The invalid algorithm is analogous to raising a B+ student's average to A, and lowering a C– average to D. To remove this bias, and to achieve voter equity, 17 sayings of Jesus rated Red (out of a total of 25) should be reclassified Pink. And 160 sayings now rated Black should be reclassified Gray. The incorrect algorithm arose from assigning equal ranges for Red:Pink:Gray:Black on the average vote scale. To weigh each vote equally, the ranges should be in the proportion 1:2:2:1. The
present page is an extensive (30k) discussion of problems with the
Jesus Seminar's voting algorithm. For a briefer (11k) discussion,
see Error in Jesus Seminar Vote Tally. The Jesus Seminar has collated the opinions of biblical scholars on the sayings of Jesus, as reported by Funk and Hoover (1993). Similar collations have been offered on the acts of Jesus (Funk 1997). The Seminar intended to give the vote of each Fellow of the Seminar equal weight in arriving at a consensus rating of Red, Pink, Gray, or Black. (See cited references for the interpretation of these colors.) Unfortunately, the Seminar devised an apparently equitable but actually biased scheme for determining the consensus of its Fellows. This paper shows how to correct that bias. "Weighted" Average Vote The Jesus Seminar has utilized a traditional, correct method for calculating a numerical indicator for each of its votes on a saying of Jesus. To begin with, the individual votes are quantized. A Fellow can vote either Red, Pink, Gray, or Black on a particular saying; in numerical terms, the Fellow can vote 3, 2, 1, or 0 but is constrained from voting 2.5 or 0.7. Next, the Seminar calculates what it calls a "weighted" average. This is done most simply as follows: • Add up all the numerical votes and divide by the number of votes. A mathematician or a computer programmer would convert this process to a formula: Average Score = (3R + 2P + G + 0)/N (1) Here R, P and G are the number of Fellows voting Red, Pink, and Gray respectively. What has happened to the Black votes? Every Black vote gets a score of zero, so the Black voters are represented by a zero in the numerator in the equation. The Black influence is felt downstairs, in the denominator of the equation. Here we find N, the total number of votes, given by N = R + P + G + B (2) B is the number of Black votes. As more Fellows vote Black, N goes up and the Average Score goes down, closer to zero which represents the Black viewpoint. Each Black vote contributes just as much as each other vote. So
far we are working with an Average Score that can range from 3 (pure
Red) down to 0 (pure Black). Let's stick with the 3–0 scale
for now. It's easiest to stay on the straight and narrow using the
3–0 scale. The Average Score is not a weighted average in the conventional sense. Suppose we have we have 1 Red, 3 Pink, 2 Gray, and 4 Black votes—10 votes total. If we want to talk about "weighting," then imagine that only four votes are cast. In this example, we could give a weight of 1 to the Red vote; the Pink vote would have a weight of 3; Gray would be weighted 2 and Black would be weighted 4. In Equation (1), we could consider the scores 3, 2, 1, and 0, with R, P and G as the weights. This is the normal mathematical meaning of weighting: to calculate an average score one multiplies each score by its corresponding weight. But the Jesus Seminar chose to talk about 3, 2, 1, and 0 as the weights (Funk, 1986). This is not the normal nomenclature. In fact, we have 10 votes in our case, not four. Using the sum-up-all-the-votes rule, we obtain the Average Score: (3 + 2 + 2 + 2 + 1 + 1 + 0 + 0 + 0 + 0)/10 = 11/10 = 1.1 (3) In this calculation, we use the principle • Each vote has equal weight. There are no extra factors multiplying each score. Each score is counted only once. So from the perspective of the individual votes (which we shall follow), the scoring is not weighted. The equal weighting, that is, the lack of weighting of each vote shows the (intended) democratic character of the Jesus Seminar, at least as far as the calculation of a numerical Average Score is concerned. Some
analogies may be easier to follow than these equations. Let's imagine that all the Seminar Fellows try to balance on a see-saw. In this analogy, there are several rules:
* Each Fellow weighs the same (the principle of equal weighting). Let's consider an example, with reduced total numbers for convenience. Suppose 10 Fellows vote Red and 10 vote Pink. Figure 1 shows the see-saw for this case.
10 10 0 0 Figure 1. See-saw for Red/Pink balance (3–0 scale). Using Equation (1), one has Average Score = (3•10 + 2•10)/20 = 2.5. But you could get the average even more easily by looking at the see-saw. It seems obvious that to balance it one should put the fulcrum (balance point) at 2.5. Let's make the situation more complicated. Add 5 Red votes. That would raise the Average Score up to 2.6, according to Equation (1). How many Black votes are needed to bring the average back down to 2.5? You don't need to use the equation; just consider the see-saw shown in Figure 2.
15 10 0 1 Figure 2. See-saw for Red/Pink balance, showing leverage of Black vote (3–0 scale). Because the Black voter is sitting five times as far away from the fulcrum as the Red voters, you can see from high school physics that a Black vote has five times the leverage as a Red vote. So one Black vote balances out five Red votes, when the fulcrum is at 2.5. Similarly,
if the votes were mostly Gray and Black, one Red vote would balance
five Black votes, if the fulcrum were at 0.5. If the fulcrum were
at 1.5, one Red would balance three Grays, and one Black would balance
three Pinks. Try it out! As Funk and Hoover observe in The Five Gospels, "Since most of the Fellows of the Seminar are professors, they are accustomed to grade points and grade-point averages" (p. 37). Indeed, there is an exact analogy between calculating grade point averages and calculating average scores for the Seminar. The analogy works only if we impose these rules:
* In each course, a student is given a grade of 4, 3, 2, or 1, corresponding
to A, B, C, and D. No grades of F, 3.5 or C– are allowed. The grade point average is determined by adding up all the grades and dividing by the number of courses. One gets a result ranging from 4 to 1. The only difference from calculating Seminar scores is that GPAs are on a 4–1 scale instead of a 3–0 scale. Suppose a student has 5 A's and 5 B's at the end of the sophomore year. The GPA would be 3.5. If the student works extra hard as a junior and gets 5 more A's but also gets one D, the GPA is still stuck at 3.5. The analogy is exact with our last see-saw case. As Funk and Hoover remark, one low grade among several A's "could readily pull an average down" (p. 37). Announcing Quantized Voting Results If the Average Scores of Jesus Seminar votes were rounded off to the nearest tenth, then one could have average values of 3.0, 2.9, 2,8, . . . down to 0.1, 0.0. For The Five Gospels to reflect these shades of opinion, it would have to be printed using a palette of 31 different colors. This would give the printer nightmares, and would give a false impression of precision to this inexact science. Consequently, the Jesus Seminar decided to express the consensus of its votes in a quantized fashion. That is, the overall opinion of the Fellows was expressed by rating each saying of Jesus Red, Pink, Gray or Black. After all, the individual votes are similarly quantized; the same should go for the corporate opinion. There is a right way to do the quantization and a wrong way. The Correct Algorithm (3–0 Scale) There is a very simple algorithm for converting Average Scores like 2.2 or 1.3 into only four possible values: • Round off each Average Score to the nearest integer. Note that this algorithm works only on the original 3–0 numerical scale. This leads us to the round-off algorithm presented in Table 1. Table 1. Correct algorithm for 3–0 scale
Average Rounded What if the Average Score is exactly 2.50, or 1.50, or 0.50? Here one needs to invoke a special rule. Scientists usually round off to the nearest even number. When you fill out your Form 1040, you find that greedy Uncle Sam insists that you round every 50 cents of your income up to the nearest dollar. The Jesus Seminar chose to round all ties downward; this negative bias affects very few of the votes. There is an alternative algorithm for converting raw scores into four colors: • Use tie vote scores as break points. If the Fellows of the Seminar were voting only Red and Pink, the Average Score would be above 2.5 if the majority voted Red, equal to 2.5 in the case of a tie, and below 2.5 for a Pink majority. So the algorithm in Table 1 just above would give make the appropriate decisions in the cases of Red/Pink, Pink/Gray, and Gray/Black split votes. Table 2. Correct tie vote scores (3–0 scale)
Average
We can apply the see-saw analogy to the correct algorithms in Tables 1 and 2. To make a Red-vs-Pink test, put the fulcrum at 2.5, as shown in Figures 1 and 2. In the two cases shown, the see-saw balances, showing a weighted tie vote. If the see-saw tilts down at the left, then Red has won the vote. If the see-saw tilts down at the right, then Pink (or perhaps Gray or Black) has won. Looking at Figures 1 and 2, with the fulcrum located at 2.5 for a Red-vs-Pink test, you can see that Red and Pink votes have equal leverage. However, a Fellow sitting at the Gray location has three times the leverage of a Fellow at the Red location. And, as in Figure 2, a Black vote has five times the leverage of a Red vote. Yet each Fellow has equal weight in the see-saw analogy. What happens in a Pink-vs-Gray test? (For some gospel scholars, this is the most important test.) Suppose there are 5 Red votes and 15 Gray votes. The see-saw balances, as shown in Figure 3.
5 0 15 0 Figure 3. See-saw with fulcrum located at 1.5 for Pink-vs-Gray test (3-0 scale). This figure shows graphically that in a Pink-vs-Gray test each Red vote has three times the leverage of each Gray vote. You can easily see from the same figure that each Black vote has three times the leverage of each Pink vote when the fulcrum is put at 1.5. To make a Gray-vs-Black test, put the fulcrum at 0.5. We obtain Table 3 for the relative leverage of different votes in different tests, according to the correct decision- making algorithm. Table 3. Leverage of votes for various tests (correct algorithm)
Relative leverage
The grade point average analogy works well here, if one invokes the rule that the registrar must round off all announced GPAs to the integers 4, 3, 2, or 1. The registrar would use the algorithm of Table 4. Table 4. Correct algorithm for announcing grade point averages
Raw Announced Raw Announced
It is remarkable that the Pink and Gray ratings enjoy a range of average scores that is twice that of the Red and Black ratings, as shown in Table 1. Can that be fair? Yes, it is. The four ratings do not have equal environments. Pink and Gray are midspace ratings, and have extra space above and below them. Red is a ceiling vote; there is no space above. Black is a floor vote; one can't go further down. (If the Fellows could have voted . . . –2, –1, 0, 1, 2, 3, 4, . . . on each saying, and we went through the same kind of exercise, then the environments for 0, 1, 2, and 3 would be similar, and we would obtain equal ranges for the 0, 1, 2, and 3 announced votes.) The unequal ranges also appear in Table 4, for the grade point averages. Because no course grades of A+ or D– are given out, the range of raw GPAs is less for A's and D's than for B's and C's. In both the Seminar voting and GPA calculations, the input ranges for the four possible outputs have the ratios Red:Pink:Gray:Black = A:B:C:D = 1:2:2:1. The ranges are not equal. Yet all votes (or grades) are equally weighted. The Incorrect Algorithm (3–0 Scale) In attempting to devise an scheme for determining announced color ratings, the Jesus Seminar invoked the incorrect principle that all Average Score ranges should be equal, on the grounds that this scheme would provide equity between votes of the four colors. The scheme presented in The Five Gospels is based on dividing original scores by a factor of three. Multiplying by three to get back to the original 3–0 scale, we have the scheme of Table 5. Table 5. Incorrect scheme for announcing consensus votes (3–0) scale)
Average Rounded This scheme provides equal ranges for the four final color ratings, but there is no mathematical basis for the equal range concept. And the incorrect scheme does violence to common sense. Suppose the Fellows of the Jesus Seminar were equally divided between Red (3) and Pink (2), with no votes for Gray or Black. Then the Average Score would be 2.5, and the vote would be given (according to Table 5) a solid Red rating! Suppose that there was one more Pink vote than Red; the raw score would be about 2.4, but the verdict would still be Red, in contravention of the will of the majority of the Fellows. Consider an extreme case. Suppose that 100 Fellows were present and voting. Suppose 74 votes are for Gray, and 26 votes are for Black. Then the Average Score would be 74/100 = 0.74, which lies below the incorrect Gray/Black break point in Table 5, and the consensus of the Seminar would be announced as Black. It is evident that providing equal ranges for the four colors leads to inequity. The inequity can be readily displayed using the see-saw. For a Gray-vs-Black test, the invalid algorithm would put the test fulcrum at 0.75. To balance the see-saw about that point, we can seat 75 Fellows at the Gray point and 25 at Black. The see-saw balances!
0 0 75 25 Figure
4. See-saw for Gray-vs-Black test, balanced on the fulcrum incorrectly
It is evident that the incorrect algorithm has the effect of giving each Black vote three times the weight of each Gray vote. The algorithm violates the equal weight rule. Similarly, in Figures 1 and 2, if we incorrectly located the fulcrum at 2.25 for a Red-vs-Pink test, you can see that we would be giving every Red vote three times the weight of every Pink vote. To use the see-saw for a Pink-vs-Gray test, either algorithm instructs to put the fulcrum at 1.5, which is correct. The grade point average analogy also shows the flaws of the Jesus Seminar's algorithm. If we add 1 to the scores in Table 5, we have a scheme for determining rounded-off GPAs, as seen in Table 6. Table 6. Incorrect scheme for announcing grade point averages
Raw Announced Raw Announced The GPA scheme shown in Table 6 is the exact analogue of the voting algorithm used by the Jesus Seminar. The scheme follows the Seminar's cardinal rule of equal ranges. But look at what it does! If a student had a GPA of 3.26, the Registrar would round that up to 4. The Registrar would round a GPA of 1.74 down to 1. A GPA of B+ would be announced to the world as an A, and a C– average would be converted to a D. That's the kind of thing that the Jesus Seminar's tally clerk has been doing for the sayings of Jesus. A number of examples of the failing of the incorrect scheme are presented below. Algorithms for the 1–0 Scale Converting all scores from the 3–0 scale to a 1–0 scale, as was done by the Jesus Seminar, is valid as an abstract mathematical operation. However, the transformation makes it easier to commit errors. The flaws in the Seminar's scheme are less obvious on the 1–0 scale than on the 3–0 scale. The Correct Algorithm (1–0 Scale) Dividing all numbers in Table 1 by three leads to Table 7. Table 7. Correct algorithm for 1–0 scale
Average Announced This
algorithm displays the same ratio of ranges, Red:Pink:Gray:Black =
1:2:2:1, as we had before with the 3–0 scale. This algorithm
preserves the equity among all voters, however. Again consider a Red/Pink
tie. The Average Score is 2.5 on the 3–0 scale, which converts
to 2.5/3 = 0.833 on the 1–0 scale. So if Red has a slight ascendancy,
this algorithm will award a Red rating; if Pink has the edge, then
Pink will be announced. The reader is probably already aware of the customary incorrect algorithm of the Jesus Seminar, shown in Table 8. Table 8. Incorrect algorithm for 1–0 scale
Average Announced
The break point between Pink and Gray is the same for both sets of algorithms. Correcting the Voting Record To properly show the opinions of the Fellows of the Jesus Seminar, all votes resulting in an average in the 0.750–0.833 range, which have heretofore been erroneously printed Red, should be rated Pink. And all vote averages in the 0.167–0.250 range, which have been wrongly rated Black, should be reclassified Gray. (We will use only the 1–0 scale from now on.) The Five Gospels (in a table at the very end of the book) shows the numerical voting results for sayings of Jesus that were printed in Red or Pink; that is, that had average votes from 1.000 to 0.500. 25 were rated Red. Only 8 of these ratings reflect the views of the Seminar; 17 sayings were rated Red but the Fellows actually voted them Pink (average vote fell in the 0.833–0.750 range). This conclusion is confirmed, for the five gospels, from the more detailed tables of "Voting Records" in Foundations and Facets Forum, vol. 6 (June 1990.) For example, the following sayings should have ratings changed from Red to Pink: • "Render to Caesar the things that are Caesar's and to God the things that are God's" (Mark 12:17 KJV). • The story of the Good Samaritan (Luke 10:30–35). The same Forum article also shows the detail for votes having averages below 0.500—that is, for Gray and Black ratings. For the five gospels, the misrated sayings are listed in Table 9. Table 9. Sayings rated Black but actually voted as Gray
Mark 22
These items include the confirmation of the Two Great Commandments (Luke 10:28b), the Golden Rule (Thomas 6:2), and blessings to the merciful, pure in heart, and peacemakers (Matthew 5:7–9 NRSV). Outstanding Examples For the erroneous algorithm in action, one turns to the pages of Forum, vol. 6 (June 1990). Consider the following:
%Red %Pink %Gray %Black Avg. Color Neglecting the Gray votes, the balance between Red and Pink was definitely in favor of Pink. The Gray votes pull the consensus even farther into the Pink zone. One wonders why the Fellows who voted Pink and Gray did not object to this saying being rated Red—in accordance with the faulty algorithm. Putting the Mustard Seed parable to the see-saw test, we get Figures 5 and 6.
39 50 11 0 Figure
5. See-saw for incorrect Red-vs-Pink test on Mustard Seed parable
(Thomas 20). The incorrect algorithm of the Jesus Seminar requires that the fulcrum be put at 2.25. That would cause the left of the see- saw to tilt down, and the right to tilt up, because the faulty algorithm favors Red votes over Pink votes by a three-to-one ratio. The correct test is shown in Figure 6.
39 50 11 0 Figure
6. See-saw for correct Red-vs-Pink test on Mustard Seed parable (Thomas
20). With the fulcrum at its correct location (2.5), the see-saw tilts down at the right, as it should for equal weights to all votes, and Pink wins the contest. There are several sayings where the misrepresentation of the opinion of the Fellows by the erroneous algorithm is obvious (the last column of Table 10 shows the final rating by the Seminar, as printed in The Five Gospels): Table 10. Sayings voted Gray but rated Black %Red %Pink %Gray %Black Avg. Color Doctor,
cure yourself Bread
and leaven Calculating
cost: Tower, king Finding
the world Who
has seen the wind? Reborn
to enter the kingdom Leaven
of the Pharisees Horses
and bows
In all these cases (and in all Gray-vs-Black tests with average scores in the 0.25–0.167 range) the algorithm used by the Seminar has the effect of counting each Black vote three times but each Gray vote once, invalidly throwing the contest to a Black rating. Significance of the Error If your chief concern is to classify the sayings of Jesus into two bins—(Red-Pink) and (Gray-Black)—then it doesn't matter which algorithm you use. Both algorithms agree on the break point between Pink and Gray. The errors in the algorithm are significant only if, as a Fellow of the Seminar, you devoted considerable energy to choosing between a Red or Pink vote, or between a Gray or Black vote. If those distinctions were important to you, then it is important to correctly present your opinion to the rest of the world. The errors might be important if you seek to embrace only those sayings of Jesus whose authenticity is unchallenged by a critical group of scholars. The Five Gospels offers 25 such (Red) sayings; the present considerations have pruned the list down to only 8. The Gray-vs-Black distinction is not trivial. A Gray vote can signify that Jesus may well have uttered the saying, but
* it was not original with Jesus, or These characteristics are quite different from a Black (No, He Didn't Say It) vote. References Robert W. Funk, "Poll on the Parables," Foundations and Facets Forum, 2,1 (1986) 56–57. Robert W. Funk and the Jesus Seminar, The Acts of Jesus: The Search for the Authentic Deeds of Jesus, HarperSanFrancisco, 1998. Robert W. Funk, Roy W. Hoover, and the Jesus Seminar, The Five Gospels: The Search for the Authentic Words of Jesus, Macmillan, New York, 1993. The Jesus Seminar, "Voting Records: Sorted by Weighted Average," Foundations and Facets Forum, 6,2 (1990) 139–91. Lee A. Young, PhD 144 Chestnut Circle Lincoln, Massachusetts 01773 leeayoung@aol.com 781-259-9563 What does Lee Young do when he is not worrying about the mathematics of voting algorithms? He does his own theology, as found on his web page at Where Can I Find God? (If you get an error message, try clicking on RELOAD.) Posted June 7, 1998. Revised February 10, 2000. http://members.aol.com/leeayoung/jsemnar2/algrithm.htm This page has been accessed times. Copyright © 2000 by Lee Adams Young. |
NOTE: I am hosting this page on this site only because I cannot find a copy of it on the web outside of archives. It originally appeared at the following URL http://members.aol.com/leeayoung/jsemnar2/algrithm.htm |