Doubling the Canon 2021 - Improvement suggestions. NAME POLL!

[Angel Glez]

As you know there are some changes planned for next update. It will also be a right moment to introduce improvements in our game.

Name: Doubling the Canon
Host: Angel Glez
Canon: TSPDT GF top 1000 + TSPDT 21st Century top 1000 + TSPDT GF top 1001-2000
Doubling list: 1100 1000 titles
Nominations: 12 films per doubler + TSPDT drop-offs + DtC holdovers
Rating system: 0-6
Adjustment for popularity: For every 10 votes the lowest vote gets dropped (???)
Threshold: 8 votes
Re-voting: After 5 years

EVERYTHING is open to discussion, but keep in mind that what we are mainly looking for is a friendlier game, higher participation and more film discussion and discovery. You've got the floor!

[sol]

Name: Doubling the Canon
Host: Sir Angel Glez, Esq.
Canon: TSPDT GF top 1000 + TSPDT 21st Century top 1000 + TSPDT GF top 1001-2000
Doubling list: 1100 1200 titles
Nominations: 12 15 films per doubler + TSPDT drop-offs + DtC holdovers
Rating system: 0-6
Adjustment for popularity: For every 10 votes the lowest vote gets dropped (not needed if [see below])
Threshold: 8 votes - 20% of the total number of participants*
Re-voting: After 5 years

* This would create a truer consensus list as opposed to a niche list of obscure favourites, which I believe is more in line with Lee Price's original vision.

I also think that it is problematic that a 3-vote is meant to be "good enough for the bottom half of the canon" and yet 3.90 is the cut-off - which is why I think a bigger list is better. Again, in line with Lee's original vision as I understood it.

Just my two cents; feel free to ignore...

[mjf314]

One possible change is replace "adjustment for popularity" with the bayesian estimate formula, to penalize films with fewer votes. I posted a sample list here:
viewtopic.php?p=640215#p640215
The numbers can be tweaked if you want a bigger or smaller penalty.

The films near the top will be films that a lot of people like. Films with few votes can still make the list, but they need a higher rating. The list above contains a few films rated 6.0 with 3 votes, but if you want to get rid of them you can increase the penalty.

I'm not sure about the list size, but the list size can be decided after the voting is finished.

I don't really like the idea of a constant minimum. For example, suppose there are 2 films:
Film A is rated 5.5 with 7 votes.
Film B is rated 3.9 with 8 votes.
Surely film A deserves to be on the list more than film B.

@sol: 20% of total participants would have been 18 this year. That seems to me like too much.

[edward5]

Some voters have a high average rating. And some voters have a low average rating.
it is because of different evaluation criterion.
So I think everyone's rating need to adjust.

[mjf314]

Some voters have a high average rating. And some voters have a low average rating.
it is because of different evaluation criterion.
So I think everyone's rating need to adjust.

Do you have a suggestion of how to do this?

[zzzorf]

Most of everything seems fine to me, I just like to participate as I managed to find a couple of great movies in the nominations this year. My only thought would be the name. With the Canon having grown larger we aren't really Doubling it anymore. Maybe changing it to something like "Extending the Canon" may better suit the new landscape?

[AdamH]

Is there a post that very clearly lays out exactly how Doubling the Canon works? Voting, re-voting, holdovers, nominations etc.? I'd personally find that very useful as the seemingly complicated system has always put me off.

[Melvelet]

One possible change is replace "adjustment for popularity" with the bayesian estimate formula, to penalize films with fewer votes. I posted a sample list here:
viewtopic.php?p=640215#p640215
The numbers can be tweaked if you want a bigger or smaller penalty.

The films near the top will be films that a lot of people like. Films with few votes can still make the list, but they need a higher rating. The list above contains a few films rated 6.0 with 3 votes, but if you want to get rid of them you can increase the penalty.

I'm not sure about the list size, but the list size can be decided after the voting is finished.

I don't really like the idea of a constant minimum. For example, suppose there are 2 films:
Film A is rated 5.5 with 7 votes.
Film B is rated 3.9 with 8 votes.
Surely film A deserves to be on the list more than film B.

@sol: 20% of total participants would have been 18 this year. That seems to me like too much.

I really like this. And if we define a sensible way to determine the holdovers (a list that penalizes few votes (way) less and determining a good length for the holdover list), we'd keeping having an interesting spotlight for the niche films.

[brokenface]

This is just throwing something out, but would there be a possibility of making the vote a 2-stage thing, so rather than having holdovers for the following year, you first whittle down the big list of 1300+ nominees to say 200 films using some combination of lower threshold of rating/lower # of votes and then you have a smaller ballot where people might have time to watch some of the ones on the borderline that same year, rather than them essentially starting from scratch the following year. Also the 2nd ballot would be a bit less overwhelming to go through so you might get more casual participants joining at that point.

I'm counting myself as casual as I haven't participated for a few years just through not getting round to going through the ballot of 1200+ films. I can go through with IMDB open and try to translate my ratings, but I do find it a bit of a cumbersome process to get through & I'm never sure I'm being fair to films as it's hard to calibrate that many votes.

[mjf314]

I really like this. And if we define a sensible way to determine the holdovers (a list that penalizes few votes (way) less and determining a good length for the holdover list), we'd keeping having an interesting spotlight for the niche films.

If we add 4 2-votes to each film:
The holdovers can be every film with fewer than 8 votes AND a bayesian rating of 3.27 or higher.

If we go with a larger penalty and add 5 2-votes to each film:
The holdovers can be every film with fewer than 8 votes AND a bayesian rating of 3.14 or higher.

Either way, the holdover cutoffs would be approximately the same as they are now.

[sol]

@sol: 20% of total participants would have been 18 this year. That seems to me like too much.

It depends what you see the purpose of the list as being.

When lee-109 first proposed the project circa 2005, I recall talk about creating a consensus canon of films that we collectively thought were worthy enough to be in the official TSP canon.

20% could be lowered to 15% - but then do we really have a consensus canon when the list is full of films that 85% of participants have never seen?

I really like this though, which would seem to resolve some of my reservations about how the game has evolved since its inception:

One possible change is replace "adjustment for popularity" with the bayesian estimate formula, to penalize films with fewer votes. I posted a sample list here:
viewtopic.php?p=640215#p640215
The numbers can be tweaked if you want a bigger or smaller penalty.

The films near the top will be films that a lot of people like. Films with few votes can still make the list, but they need a higher rating.

[OldAle1]

When lee-109 first proposed the project circa 2005, I recall talk about creating a consensus canon of films that we collectively thought were worthy enough to be in the official TSP canon.

The thing is though, in 2005 the "canon" was 1000 films, and now it's 3000. BIG difference. How many films out there are there that could be in any was considered "consensus" choices beyond those 3000? While I tend to agree with you overall, and do find too much obscurity on the list now, I'm not sure how to avoid that and highlight films that are really loved, especially with all the obscurity-chasers and mainstream-haters that make up at least half of the voters.

[St. Gloede]

I would love to see DtC drop-off revotes as well.

The popularity scoring seems pretty good to be honest, maybe tweak slightly further, i.e. for every 5 votes over X threshold - unless that gets too complicated.

[72aicm]

Exlude TSPDT 1001-2000. No need to change it other than that. I love that list and the quirky, and sometimes unfair voting system.

[joachimt]

Set the list length to 1000. I never understood how the length has been determined.

[AdamH]

Set the list length to 1000. I never understood how the length has been determined.

Definitely agree with this. Anything over 1000 is too long (especially as I think it was up to 1300+ at one point). Even 1000 is a huge list.

[ChrisReynolds]

One possible change is replace "adjustment for popularity" with the bayesian estimate formula, to penalize films with fewer votes. I posted a sample list here:
viewtopic.php?p=640215#p640215
The numbers can be tweaked if you want a bigger or smaller penalty.

Could you share the formula you used?

I think moving to this method would produce a better list, especially as the current balancing method (minimum number of votes, lowest vote gets dropped for every ten votes) seems crude and creates weird situations where a film could have got on the list by one extra person voting a 1.

Other than that I would support limiting the list to 1000 films and keeping the films people are allowed to nominate at 12.

[sol]

I must be the only user who finds longer lists more interesting. The longer the list, the more fun (i.e. challenging) it is to complete. Also, with more films, there is a greater likelihood of coming across films that have not already been placed on my radar thanks to other lists.

[Angel Glez]

Is there a post that very clearly lays out exactly how Doubling the Canon works? Voting, re-voting, holdovers, nominations etc.? I'd personally find that very useful as the seemingly complicated system has always put me off.

Actually what we ask the doubler is not complicated at all. In Phase I we only need 12 films that he thinks are great but are not on a given list (the Canon). In Phase II he must rate the films he has seen from the roster of nominees. Everything else is a intrinsic process that he cannot control.

[mjf314]

One possible change is replace "adjustment for popularity" with the bayesian estimate formula, to penalize films with fewer votes. I posted a sample list here:
viewtopic.php?p=640215#p640215
The numbers can be tweaked if you want a bigger or smaller penalty.

Could you share the formula you used?

I think moving to this method would produce a better list, especially as the current balancing method (minimum number of votes, lowest vote gets dropped for every ten votes) seems crude and creates weird situations where a film could have got on the list by one extra person voting a 1.

Other than that I would support limiting the list to 1000 films and keeping the films people are allowed to nominate at 12.

(rating * num_ratings + 2 * 4) / (num_ratings + 4)

It's the formula that the IMDb top 250 uses, but with different numbers. The 2 is the "average rating" (which I chose arbitrarily), and the 4 is how many average ratings I added to each film. Both numbers can be tweaked to make the list more or less obscure.

I used the adjusted ratings because I didn't have the unadjusted ratings, but it's probably better to use the unadjusted ratings.

[blocho]

I suggest a higher threshold because the rules as they are encourage obscurity. Nominators at this point generally know that they have a much better chance if they nominate avoid anything that will get more than 25 votes.

I also suggest making an adjustment to the rating system. 0-6 is a seven-point system, which is already kind of tough. Furthermore, the descriptions for each point in that system are hard to parse. How should I know a movie would be between my 600th and 1200th favorite movie? It forces me to have every movie I've ever seen ranked. Also, a movie you would put as a 5 means very different for things for someone who has seen 3000 movies versus 8000 movies. I would suggest using instead a 1-5 scale. That's a five-point scale, which matches well with the imdb 10-point scale that many people use (to adjust, you just divide your imdb rating by 2).

[PeacefulAnarchy]

One possible change is replace "adjustment for popularity" with the bayesian estimate formula, to penalize films with fewer votes. I posted a sample list here:
viewtopic.php?p=640215#p640215
The numbers can be tweaked if you want a bigger or smaller penalty.

Could you share the formula you used?

I think moving to this method would produce a better list, especially as the current balancing method (minimum number of votes, lowest vote gets dropped for every ten votes) seems crude and creates weird situations where a film could have got on the list by one extra person voting a 1.

Other than that I would support limiting the list to 1000 films and keeping the films people are allowed to nominate at 12.

(rating * num_ratings + 2 * 4) / (num_ratings + 4)

It's the formula that the IMDb top 250 uses, but with different numbers. The 2 is the "average rating" (which I chose arbitrarily), and the 4 is how many average ratings I added to each film. Both numbers can be tweaked to make the list more or less obscure.

I used the adjusted ratings because I didn't have the unadjusted ratings, but it's probably better to use the unadjusted ratings.

Changing the "average rating" makes absolutely no difference at all in the rankings, it just changes what the displayed "average" is. Only how many makes a difference.

[Fergenaprido]

My thoughts.

• Name: Doubling the Canon yes, it's no longer a strict doubling, but that doesn't really bother me - if we want to keep DTC it could be changed to "Deepening the Canon" or something similar
• Host: Angel Glez
• Canon: TSPDT GF top 1000 + TSPDT 21st Century top 1000 + TSPDT GF top 1001-2000
• Doubling list: 1100 1000 titles I definitely think that it should go back down to 1000 films; makes it more manageable, less intimidating, and now is the perfect time since we're planning to exclude the top 1001-2000 films from the list
• Nominations: 12 films per doubler + TSPDT drop-offs + DtC holdovers
• Rating system: 0-6 I like this rating system, but perhaps it needs to be adjust so that 6 is Top 500, 5 is 501-1000, etc.
• Adjustment for popularity: For every 10 votes the lowest vote gets dropped Maybe instead of every 10 votes, it's every X votes, where X is the threshold
• Threshold: 8 votes How many voters were there this year? I generally like the idea of a 10% threshold, based on the number of participants from the year before
• Re-voting: After 5 years

I'm not sure how I feel about the Bayesian proposal. I know the participants don't have to understand the mechanics of how it all works behind the scenes in order to participate, but it should be relatively clear/transparent to ensure buy-in and legitimacy. I wouldn't use arbitrary numbers, though, I would use actual averages from the process, perhaps find the average (unadjusted) rating for all films and the average number of votes for all films, and then use a rolling average of the past 5 cycles to allow for more consistency. I think that would be fairer and clearer to people than arbitrarily picking 2 and 4 as in your example.

Also, I just did a cross-reference and there are 240 films on both DTC and TSPDT 1001-2000 right now. So restricting DTC to a Top 1000 will still likely result in a lower cutoff from this year's 3.90 (I'm guessing around 3.70-3.75), which is a good thing to me, since films in the high 3s almost 4s are still ones that an average of the voters think should be on the list.

I also suggest making an adjustment to the rating system. 0-6 is a seven-point system, which is already kind of tough. Furthermore, the descriptions for each point in that system are hard to parse. How should I know a movie would be between my 600th and 1200th favorite movie? It forces me to have every movie I've ever seen ranked. Also, a movie you would put as a 5 means very different for things for someone who has seen 3000 movies versus 8000 movies. I would suggest using instead a 1-5 scale. That's a five-point scale, which matches well with the imdb 10-point scale that many people use (to adjust, you just divide your imdb rating by 2).

I don't think the system is too hard to parse, especially among a community consisting largely of chronic listmakers
I think it's okay if a 5 for a 3000-film person is different from an 8000-film person. Imdb 10-point scale is also used differently by people, so there's no consistency there if that's what you're looking for. At least the current 7-point scale clearly defines what each number means.

[PeacefulAnarchy]

I also suggest making an adjustment to the rating system. 0-6 is a seven-point system, which is already kind of tough. Furthermore, the descriptions for each point in that system are hard to parse. How should I know a movie would be between my 600th and 1200th favorite movie? It forces me to have every movie I've ever seen ranked. Also, a movie you would put as a 5 means very different for things for someone who has seen 3000 movies versus 8000 movies. I would suggest using instead a 1-5 scale. That's a five-point scale, which matches well with the imdb 10-point scale that many people use (to adjust, you just divide your imdb rating by 2).

To the first point, you don't need everything ranked, just to have some rough idea of where things fall. If you have 100 10s, 500 9s and 1000 8s, then your 8s might be in one category or the other but you can just ask "is this a really strong 8 or low 8" to parse the difference.
As for the different number of movies watched, that's a judgement that voters should be making, though I don't know if they do. When I had seen much fewer films I took that into account, so I considered my top 300 at the time to be roughly what my top 600 would be if I filled all my blind spots, now my adjustment is more 500=600 and 1000=1200 since I've seen more of the established canon. So while it's personal in the sense of it being my opinion and my ratings, the numbers for the tiers have always been about my "ideal self" who would have watched all relevant films.
I think the fact that conversion is not just imdb rating over 2 is actually something very positive about this since it means that even if you're just converting imdb ratings you have to put a little thought into what they mean. Imdb ratings can mean very different things, some people give 7 to a film they love, others to a film they like and others to a film they feel indifferent to. Converting your personal scale to DtC should require consideration of how your personal scale works.

[jeff_v]

Or you can just rate everything you've seen on a 0-100 scale in Criticker, which makes stuff like this easy.

[edward5]

V0 = numbers of film rating 0
V1 = numbers of film rating 1
V2 = numbers of film rating 2
V3 = numbers of film rating 3
V4 = numbers of film rating 4
V5 = numbers of film rating 5
V6 = numbers of film rating 6

T = Total votes
T = V0+V1+V2+V3+V4+V5+V6

M0 = Median of V0
M0 = (1+V0)/2
M1 = (1+V1)/2
M2 = (1+V2)/2
M3 = (1+V3)/2
M4 = (1+V4)/2
M5 = (1+V5)/2
M6 = (1+V6)/2
Mn = (1+Vn)/2

Cn = from 0 to Mn
C0 = M0 = (1+V0)/2
C1 = V0+M1 = V0+(1+V1)/2
C2 = V0+V1+M2 = V0+V1+(1+V2)/2
C3 = V0+V1+V2+M3 = V0+V1+V2+(1+V3)/2
C4 = V0+V1+V2+V3+M4 = V0+V1+V2+V3+(1+V4)/2
C5 = V0+V1+V2+V3+V4+M5 = V0+V1+V2+V3+V4+(1+V5)/2
C6 = V0+V1+V2+V3+V4+V5+M6 = V0+V1+V2+V3+V4+V5+(1+V6)/2
Cn = V0+V1+...+V(n-1)+Mn = V0+V1+...+V(n-1)+(1+Vn)/2

average rating overall of 2020 = 3.19
G = perfect film rating = 6.38


New Score[adjust] = N
N0 = C0*G/T = C0*6.38/T
N1 = C1*G/T = C1*6.38/T
N2 = C2*G/T = C2*6.38/T
N3 = C3*G/T = C3*6.38/T
N4 = C4*G/T = C4*6.38/T
N5 = C5*G/T = C5*6.38/T
N6 = C6*G/T = C6*6.38/T



i.e.

xianjiro

6 0 0%
5 7 1%
4 37 6%
3 190 31%
2 280 45%
1 96 15%
0 10 2%


V0 = 10
V1 = 96
V2 = 280
V3 = 190
V4 = 37
V5 = 7
V6 = 0

T = Total votes
T = V0+V1+V2+V3+V4+V5+V6 = 620

median
M0 = (1+V0)/2 = 5.5
M1 = (1+V1)/2 = 48.5
M2 = (1+V2)/2 = 140.5
M3 = (1+V3)/2 = 95.5
M4 = (1+V4)/2 = 19
M5 = (1+V5)/2 = 4
M6 = (1+V6)/2 = 0.5

from 0 to median
C0 = M0 = 5.5
C1 = V0+M1 = 58.5
C2 = V0+V1+M2 = 246.5
C3 = V0+V1+V2+M3 = 481.5
C4 = V0+V1+V2+V3+M4 = 595
C5 = V0+V1+V2+V3+V4+M5 = 617
C6 = V0+V1+V2+V3+V4+V5+M6 = 620.5


average rating overall of 2020 = 3.19
G = perfect film rating = 6.38

N = new score
N0 = C0*G/T = 5.5*6.38/620 = 0.06
N1 = C1*G/T = 58.5*6.38/620 = 0.60
N2 = C2*G/T = 246.5*6.38/620 = 2.54
N3 = C3*G/T = 481.5*6.38/620 = 4.95
N4 = C4*G/T = 595*6.38/620 = 6.12
N5 = C5*G/T = 617*6.38/620 = 6.35
N6 = C6*G/T = 620.5*6.38/620 = 6.39

score Vote adjust score
6 0 6.39
5 7 6.35
4 37 6.12
3 190 4.95
2 280 2.54
1 96 0.60
0 10 0.06

name	0	1	2	3	4	5	6	all scores	T=all votes	ave. score
xianjiro	10	96	280	190	37	7	0	1409	620	2.272580645
peacefulanarchy	53	28	61	52	82	43	41	1095	360	3.041666667
mjf314	4	6	18	73	72	36	36	945	245	3.857142857
matthewscott8	15	45	37	13	20	28	43	636	201	3.164179104
oldale1	15	38	102	143	113	40	36	1539	487	3.160164271
frbrown	2	14	45	52	62	34	28	846	237	3.569620253
ebbywebby	21	70	103	106	71	40	11	1144	422	2.710900474
lineuphere	6	11	28	55	49	49	20	793	218	3.637614679
sol	15	28	83	108	89	59	48	1457	430	3.388372093
lonewolf2003	8	46	42	53	104	46	11	1001	310	3.229032258
jal90	3	7	17	39	32	42	30	676	170	3.976470588
opio	0	8	20	19	32	25	18	466	122	3.819672131
fergenaprido	31	28	41	17	40	53	34	790	244	3.237704918
hurluberlu	17	36	71	139	95	52	8	1283	418	3.06937799
sortile9io	6	37	66	86	58	63	30	1154	346	3.335260116
test 1	100	100	100	100	100	100	100	2100	700	3
test 2	200	0	200	0	200	0	200	2400	800	3
test 3	0	0	0	100	200	200	300	3900	800	4.875
test 4	300	100	50	25	12	6	3	371	496	0.747983871

adjust	0	1	2	3	4	5	6	all scores	ave. score
xianjiro	0.06	0.60	2.54	4.95	6.12	6.35	6.39	1980.99	3.195145161
peacefulanarchy	0.48	1.20	1.98	2.99	4.17	5.28	6.03	1151.59	3.198861111
mjf314	0.07	0.20	0.51	1.69	3.58	4.99	5.92	784.74	3.203020408
matthewscott8	0.25	1.21	2.51	3.30	3.82	4.59	5.71	644.38	3.205870647
oldale1	0.10	0.45	1.37	2.97	4.65	5.65	6.15	1556.72	3.196550308
frbrown	0.04	0.26	1.05	2.36	3.89	5.18	6.02	759.22	3.203459916
ebbywebby	0.17	0.85	2.16	3.74	5.08	5.92	6.30	1349.37	3.197559242
lineuphere	0.10	0.35	0.92	2.14	3.66	5.09	6.10	698.61	3.204633028
sol	0.12	0.44	1.26	2.68	4.14	5.24	6.03	1374.89	3.197418605
lonewolf2003	0.09	0.65	1.55	2.53	4.15	5.69	6.28	992.09	3.200290323
jal90	0.08	0.26	0.71	1.76	3.10	4.48	5.84	545.49	3.208764706
opio	0.03	0.24	0.97	1.99	3.32	4.81	5.94	392.37	3.216147541
fergenaprido	0.42	1.19	2.09	2.85	3.60	4.81	5.95	781.55	3.20307377
hurluberlu	0.14	0.54	1.36	2.96	4.75	5.87	6.33	1336.61	3.197631579
sortile9io	0.06	0.46	1.41	2.81	4.14	5.26	6.11	1106.93	3.199219653
test 1	0.46	1.37	2.28	3.19	4.11	5.02	5.93	2236.19	3.194557143
test 2	0.80	1.60	2.40	3.19	3.99	4.79	5.59	2555.19	3.1939875
test 3	0.00	0.00	0.00	0.40	1.60	3.19	5.19	2555.19	3.1939875
test 4	1.94	4.51	5.47	5.96	6.19	6.31	6.37	1585.43	3.196431452

[PeacefulAnarchy]

Something about that feels very gameable, but I'm still not quite sure what's going on. (I understand the calculations and the way you're normalizing, I'm just not sure I understand the underlying dynamics of how changing one vote or adding a vote ripples through).
One negative is that everything as it is now, despite people regularly being confused about the process, is pretty straight forward and that most definitely is not.
On the plus side no more ties.

[Fergenaprido]

I also suggest making an adjustment to the rating system. 0-6 is a seven-point system, which is already kind of tough. Furthermore, the descriptions for each point in that system are hard to parse. How should I know a movie would be between my 600th and 1200th favorite movie? It forces me to have every movie I've ever seen ranked. Also, a movie you would put as a 5 means very different for things for someone who has seen 3000 movies versus 8000 movies. I would suggest using instead a 1-5 scale. That's a five-point scale, which matches well with the imdb 10-point scale that many people use (to adjust, you just divide your imdb rating by 2).

To the first point, you don't need everything ranked, just to have some rough idea of where things fall. If you have 100 10s, 500 9s and 1000 8s, then your 8s might be in one category or the other but you can just ask "is this a really strong 8 or low 8" to parse the difference.
As for the different number of movies watched, that's a judgement that voters should be making, though I don't know if they do. When I had seen much fewer films I took that into account, so I considered my top 300 at the time to be roughly what my top 600 would be if I filled all my blind spots, now my adjustment is more 500=600 and 1000=1200 since I've seen more of the established canon. So while it's personal in the sense of it being my opinion and my ratings, the numbers for the tiers have always been about my "ideal self" who would have watched all relevant films.
I think the fact that conversion is not just imdb rating over 2 is actually something very positive about this since it means that even if you're just converting imdb ratings you have to put a little thought into what they mean. Imdb ratings can mean very different things, some people give 7 to a film they love, others to a film they like and others to a film they feel indifferent to. Converting your personal scale to DtC should require consideration of how your personal scale works.

Once again, PA found a way to say what I wanted to say more clearly

Something about that feels very gameable, but I'm still not quite sure what's going on. (I understand the calculations and the way you're normalizing, I'm just not sure I understand the underlying dynamics of how changing one vote or adding a vote ripples through).
One negative is that everything as it is now, despite people regularly being confused about the process, is pretty straight forward and that most definitely is not.
On the plus side no more ties.

Yeah, I'm also not sure I fully understand what your showcasing edward. Could you add a few lines of text to explain it?

[edward5]

average rating overall of 2020 = 3.19
so the perfect film rating = 6.38 ( this number can be changed to any number)
if we think of all the films from hate to favorite score from 0 to 6.38.
so the first film get 0 , the second film get 0.01 , the third film get 0.02 .... all the way to the last film get 6.38

use median to represent a bunch of the films get the same score
if one rate 620 films and 10 films get zore score . 96 films get one score. 280 films get two score. 190 films get three score. 37 films get four score. 7 films get five score. no film gets six score.


10 films get zore score
median of 10 is (1+10)/2 = 5.5, so all 10 films use the 5.5's film score to be the new adjustment score
the 620's film get the perfect film rating = 6.38
the 5.5's film get the rating = 5.5*6.38/620 = 0.06
so all the first ten films' adjustment score = 0.06

96 films get one score
median of 96 is (1+96)/2 = 48.5, so use 48.5's film score to be the new adjustment score
and not forget we have 10 films already adjust from score 0.
10+48.5 = 58.5 so we use 58.5's film score to be the new adjustment score for film rated 1
the 58.5's film get the rating = 58.5*6.38/620 = 0.60
so all 96 films rated 1 now have a 0.60 adjustment score.

280 films get two score
median of 280 is (1+280)/2 = 140.5
and films got zore score and one score already have adjusted. it is 10+96 = 106 films
140.5+106 = 246.5
so we use 246.5's film score to be the new adjustment score for film rated 2
the 620's film get the perfect film rating = 6.38
the 246.5's film get the rating = 246.5*6.38/620 = 2.54
so all 280 films rated 2 now have a 2.54 adjustment score.

190 films get three score
median of 190 is (1+190)/2 = 95.5
and films got zore score, one score and two score already have adjusted. it is 10+96+280 = 386 films
95.5+386=481.5
so we use 481.5's film score to be the new adjustment score for film rated 3
the 620's film get the perfect film rating = 6.38
the 481.5's film get the rating = 481.5*6.38/620 = 4.95
so all 190 films rated 3 now have a 4.95 adjustment score.

37 films get four score
median of 37 is (1+37)/2 = 19
and films got zore score, one score, two score and three score already have adjusted. it is 10+96+280+190 = 576 films
19+576=595
so we use 595's film score to be the new adjustment score for film rated 4
the 620's film get the perfect film rating = 6.38
the 595's film get the rating = 595*6.38/620 = 6.12
so all 37 films rated 4 now have a 6.12 adjustment score.

7 films get five score
median of 7 is (1+7)/2 = 4
and films got zore score, one score, two score, three score and four score already have adjusted. it is 10+96+280+190+37 = 613 films
4+613=617
so we use 617's film score to be the new adjustment score for film rated 5
the 620's film get the perfect film rating = 6.38
the 617's film get the rating = 617*6.38/620 = 6.35
so all 7 films rated 5 now have a 6.35 adjustment score.


old score Vote adjust score
6 0 6.39
5 7 6.35
4 37 6.12
3 190 4.95
2 280 2.54
1 96 0.60
0 10 0.06

old average score = 2.27
new average score = 3.19


use this method we can adjust everyone's average score to around 3.19

[Angel Glez]

https://docs.google.com/spreadsheets/d/ ... sp=sharing

Here's a editable spreadsheet with full results (before and after the current adjustment). Maybe you could show us what happens by applying your formulas.

[mjf314]

Changing the "average rating" makes absolutely no difference at all in the rankings, it just changes what the displayed "average" is. Only how many makes a difference.

Changing the average does affect the rankings. Here's an example.

Average rating = 2
Number of average ratings added = 4
5.5 rating / 10 votes = 4.5
4.5 rating / 40 votes = 4.27

Average rating = 0
Number of average ratings added = 4
5.5 rating / 10 votes = 3.93
4.5 rating / 40 votes = 4.09

When the average is 2, the first film is ranked higher. When the average is 0, the second film is ranked higher.

[edward5]

https://docs.google.com/spreadsheets/d/ ... sp=sharing

Here's a editable spreadsheet with full results (before and after the current adjustment). Maybe you could show us what happens by applying your formulas.

xianjiro's score	vote	xianjiro's adjustment score
6	0	6.39
5	7	6.35
4	37	6.12
3	190	4.95
2	280	2.54
1	96	0.6
0	10	0.06

It needs to change every individual's score. It's a lot of work. if you sent me all the votes, so i can do it in one day.

[PeacefulAnarchy]

Changing the "average rating" makes absolutely no difference at all in the rankings, it just changes what the displayed "average" is. Only how many makes a difference.

Changing the average does affect the rankings. Here's an example.

Average rating = 2
Number of average ratings added = 4
5.5 rating / 10 votes = 4.5
4.5 rating / 40 votes = 4.27

Average rating = 0
Number of average ratings added = 4
5.5 rating / 10 votes = 3.93
4.5 rating / 40 votes = 4.09

When the average is 2, the first film is ranked higher. When the average is 0, the second film is ranked higher.

Well obviously the number of ratings would have to change if you changed the "average" to make things match. My (incorrect) point was that you have two variables but you could achieve the same thing with only one variable. But now that I do some calculations it turns out I was still wrong. I'm not sure why I thought the two variables were tied. You can get close approximations with just the one, though.
Like Ave=0
and # 2.22222
is very close but not equivalent, to your first example.

[PeacefulAnarchy]

Long description of normalization

So as I think about this more my original objection was wrong, it's not very gamable. Any rating/voting system is to some extent, but relatively speaking it's rather robust and hard to manipulate.

The idea is to "standardize" people's ratings. So someone with a high average is very generous and so their "6" is not worth as much as a "6" from someone with a low average. It all balances out, roughly, because the person with a high average has more high ratings so their 6 which is worth a bit less is given to more films and vice versa.

I'm not sure how I feel about this kind of system. It assumes that the differences in average are due to how people perceive the rating system rather than due to the quality of the films and genuine opinion. There's certainly some of that, but I'm not sure it's really the biggest cause of the differences.

I will say that, no matter what, a 0 should be a 0. I realize this would break the tiering mathematically and adjustments to make it work would be less "correct" but I think they'd be more reflective of voter intent. A 6 should probably universally be a 6 too (breaking the cleanness of the math even more), though that's maybe a bit more debatable conceptually.

[mjf314]

Well obviously the number of ratings would have to change if you changed the "average" to make things match. My (incorrect) point was that you have two variables but you could achieve the same thing with only one variable. But now that I do some calculations it turns out I was still wrong. I'm not sure why I thought the two variables were tied. You can get close approximations with just the one, though.
Like Ave=0
and # 2.22222
is very close but not equivalent, to your first example.

It depends on the ratings. If the ratings are lower than it's not as close.

Average rating = 2
Number of average ratings added = 4
3 rating / 10 votes = 2.71
2.7 rating / 40 votes = 2.64

Average rating = 0
Number of average ratings added = 2.22222
3 rating / 10 votes = 2.45
2.7 rating / 40 votes = 2.56

[Angel Glez]

It needs to change every individual's score. It's a lot of work. if you sent me all the votes, so i can do it in one day.

Let me know and the file is yours, but there has to be an easier way to do this.

The bayesian formula could work, but I need to look at this more carefully (films with only 3 votes making the list? )

[mjf314]

I'm not sure how I feel about this kind of system. It assumes that the differences in average are due to how people perceive the rating system rather than due to the quality of the films and genuine opinion. There's certainly some of that, but I'm not sure it's really the biggest cause of the differences.

I was going to say something similar. edward5 is assuming that each person's average enjoyment is about the same, but ratings differ due to different scales. I don't think this assumption is correct.

Suppose there are 2 voters:
Person A watches 100 nominees, but he's very careful to only watch films that he thinks he'll enjoy, and he loves most of them.
Person B watches 100 random nominees, including many from his least favorite genres, and he thinks most of them are meh.

Person A will give a lot of high ratings, and person B will give a lot of low ratings.

edward5's formula normalizes them, so both people end up with the same average rating. That doesn't seem right to me.

[mjf314]

It needs to change every individual's score. It's a lot of work. if you sent me all the votes, so i can do it in one day.

Let me know and the file is yours, but there has to be an easier way to do this.

The bayesian formula could work, but I need to look at this more carefully (films with only 3 votes making the list? )

If you think 3 is too few, it can be tweaked. If a film has a rating of 6.0, how many ratings should need to have, in order to make the list?

[PeacefulAnarchy]

Well obviously the number of ratings would have to change if you changed the "average" to make things match. My (incorrect) point was that you have two variables but you could achieve the same thing with only one variable. But now that I do some calculations it turns out I was still wrong. I'm not sure why I thought the two variables were tied. You can get close approximations with just the one, though.
Like Ave=0
and # 2.22222
is very close but not equivalent, to your first example.

It depends on the ratings. If the ratings are lower than it's not as close.

Yeah, like I said, I messed up somewhere in my mental analysis yesterday and carried that forward. There is no way to make them scale equivalently.

It needs to change every individual's score. It's a lot of work. if you sent me all the votes, so i can do it in one day.

Let me know and the file is yours, but there has to be an easier way to do this.

The bayesian formula could work, but need to look at this more carefully (films with only 3 votes making the list? )

No, a film with only three votes wouldn't make the list, but a film with 5 might with mjf's numbers. It all depends on which numbers you pick. If you used a real average like 3 you'd need the number of missing votes to be a lot more than mjf's proposed 4 (something like 8-15 depending on how much you want to hamper low voted films) for the formula to be meaningfully useful.

[tobias]

Name: Doubling the Canon
Host: Angel Glez
Canon: TSPDT GF top 1000 + TSPDT 21st Century top 500 + TSPDT GF top 1001-2000
Doubling list: 1000 titles
Nominations: 12 films per doubler + TSPDT drop-offs + DtC holdovers
Rating system: 0-6
Adjustment for popularity: I like mjf's suggestion of the bayesian estimate. I think it should be quite lenient but more penalizing to obscure films than the current system is.
Threshold: No threshold (with the bayesian estimate it's unescesarry)
Re-voting: After 5 years

This would also make holdovers as they are now obsolete if I understand it correctly but holdovers could instead just be the 50 films that were the closest to getting in the last time around.