Welcome to the ICM Forum. If you have an account but have trouble logging in, or have other questions, see THIS THREAD.
NOTE: Board emails should be working again. Information on forum upgrade and style issues.
Podcast: Talking Images (Episode 36 released September 1st: Thumbs Up or Thumbs Down: Is Rating Films Actually Worthwhile Communication?)
Polls: 2020 (Results), 21st Century (Results), 500<400 (Sept 24th), 1967 (Sept 29th)
Challenges: Eastern Europe, Silent Era, Low Rating
Managers Cup: Round 1 Schedule, 1B: russa03 vs zzzorf vs OldAle1 (Sep 26th)

Thumbs Up or Thumbs Down: Is Rating Films Actually Worthwhile Communication? [TALKING IMAGES]

Post Reply
User avatar
St. Gloede
Moderator
Posts: 12709
Joined: May 6th, 2011, 6:00 am
Contact:

Thumbs Up or Thumbs Down: Is Rating Films Actually Worthwhile Communication? [TALKING IMAGES]

#1

Post by St. Gloede »

Image

Get out your spreadsheets fellow film nerds, this episode is dedicated to you!

It will for once be decided if the answers are truly written in the stars, or if the good old 10/10 system is more than enough.

In our effort to discover the best rating systems and make sense of some of the universe's greatest questions we will also, of course, insult every fan of Rotten Tomatoes and At the Movies.

We will also discuss some of the rarer and dear systems, such as the classic Norwegian die role, and ask: just extreme can we be.

Don't be surprised, some of us have used the 100/100 scale, and there are still active promoters of 20/20.

Above all though, we will ask the key question: Are ratings actually a worthwhile form of communication?

You Can Listen Here:

Sounder: https://talking-images.sounder.fm/episo ... letterboxd

Spotify: https://open.spotify.com/episode/4xOU7oAXHqoUqY5Bw3dZ7j

Apple Podcasts: https://podcasts.apple.com/us/podcast/t ... 0533998353

Join in the discussion!

- Do you rate films, and if so - for how long have you been doing it - and do you take ratings seriously?

- What does it mean to take ratings seriously?

- What is your rating scale of choice? (Thumbs, Stars, Die, 10, %)

- Do you think ratings communicate anything of value?

- If you see someone rate a film, say, 7 out of 10 - what information are you taking away from it.

- Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system?

- Stars vs. 10/10 (Letterboxd vs. IMDb)

- What about Die? (Norway, Sweden)

- Is 1-100 or the percentage scale just too extreme?

- What does the lowest rating mean to you - what does the max rating represent? Is max always "perfect"? Is the minimum the opposite of perfect? And then I guess the question is - what fills up each of the steps in-between 1-10, 1-5, 1-100, etc.? What does each rating mean to you?
User avatar
sol
Donator
Posts: 12388
Joined: February 3rd, 2017, 7:00 am
Location: Perth, WA, Australia
Contact:

#2

Post by sol »

NB: This episode was recorded a number of months ago (but was only recently edited). My apologies to kongs_speech (who got mentioned in the podcast) if we used the wrong pronouns while talking about her. :hug:
|iCM | IMDb | Letterboxd | Gold Derby
Image
Long live the new flesh!
User avatar
St. Gloede
Moderator
Posts: 12709
Joined: May 6th, 2011, 6:00 am
Contact:

#3

Post by St. Gloede »

sol wrote: September 1st, 2021, 2:49 pm NB: This episode was recorded a number of months ago (but was only recently edited). My apologies to kongs_speech (who got mentioned in the podcast) if we used the wrong pronouns while talking about her. :hug:
No worries, all fixed.
User avatar
OldAle1
Donator
Posts: 6461
Joined: February 9th, 2017, 7:00 am
Location: Dairyland, USA
Contact:

#4

Post by OldAle1 »

Listened to a few minutes of podcast but then had to do something - will get back to it in a while, but in the meantime...

- Do you rate films, and if so - for how long have you been doing it - and do you take ratings seriously?

Yes. I've been rating since the early 80s - I got the idea from Rolling Stone and for a long time I used the 1-5 scale they use. Do I take them seriously? Not really - but I don't think they are wholly useless either if there is a meaningful context for them.

- What does it mean to take ratings seriously?

I don't know what that would mean in the abstract; I will tell you what it means for me. It means that if Jonathan Rosenbaum gave something **** I had to see it - because I trust his opinions more than any other critic, and because I know his taste and attitudes about film well enough to know what he means. And there are other people who I've trusted to one extent or another whose ratings might mean something on their own, divorced from any writing they might have done. But it's rare.

I don't take my own ratings particularly seriously, except that I do put a *little* more thought into rating something 9/10 or 1/2 than all the middle ratings.

- What is your rating scale of choice? (Thumbs, Stars, Die, 10, %)

1-10, solely because of IMDb - I started logging ratings originally around 1999 I think? But then lost access to that account and re-started in 2005 and have stuck to it ever since.

- Do you think ratings communicate anything of value?

Without the context of knowing something about the rater, absolutely nothing whatsoever.

- If you see someone rate a film, say, 7 out of 10 - what information are you taking away from it.

Nothing, unless I know what the rater means by it. Normally I would default to thinking that 7 out of 10 means "good" but not spectacularly good - that's my rough definition for myself - but I've seen people who virtually never rate anything above 7, and at least one person I remember from IMDb days never rated anything below 8. So without actually knowing what that 7 means to that user, there's no value attached to it.

- Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system?

I have no idea, and no it's not remotely good. I almost never look at RT and this is a big part of why.

- Stars vs. 10/10 (Letterboxd vs. IMDb)

I'm just too used to IMDb and 10 points seems to work for me now.

- What about Die? (Norway, Sweden)

No idea what that is, maybe I'll come back to this after listening.

- Is 1-100 or the percentage scale just too extreme?

I can't imagine ever using that.

- What does the lowest rating mean to you - what does the max rating represent? Is max always "perfect"? Is the minimum the opposite of perfect? And then I guess the question is - what fills up each of the steps in-between 1-10, 1-5, 1-100, etc.? What does each rating mean to you?

A lower rating can indicate EITHER a poorly-made film OR a film lacking entertainment value OR both.

There is no such thing as perfection, but I think it's silly not to use the full scale. If 10 had to mean "perfect" I would have no 10s. What it means to me is sort of hard to define but essentially, it's a film that I just LOVE and typically one where I feel after the first viewing like I just want to re-watch it immediately (which I really don't do, but I think about doing so a lot). And since I don't believe in perfect, I also don't believe in totally meretricious - a 1 doesn't mean "has no value at all" because such a concept is impossible. It just means usually something that is extremely poorly made and not enjoyable at all. But then there's the so-bad-it's-good paradigm, which throws everything off and calls into question all meaning that ratings could actually have...

Nutshell:
10 - GREAT/MASTERPIECE (top 1-10 of a year usually; on my all-time top list)
9 - Excellent (typically top 10-20 of a year)
8 - Very Good (often top 20-30 of a given year)
7 - Good but not memorably so
6 - OK, more good qualities than bad (just)
5 - the flipside of 6
4 - Poor but with multiple decent elements or maybe one great one
3 - Terrible
2 - Barely watchable but SOMETHING redeemable
1 - As close to useless as can be - if there's value here I couldn't find it
User avatar
Onderhond
Posts: 7519
Joined: December 23rd, 2012, 7:00 am
Contact:

#5

Post by Onderhond »

- Do you rate films, and if so - for how long have you been doing it - and do you take ratings seriously?
Yeps. I've been doing it since 2003, ever since I joined MovieMeter. That's 18 years I guess. And yes, I do take rating seriously. They distil your overall impression of a film into a very computable and comparable piece of data. They're not perfect of course, but they serve their purpose. For one, it sure makes my yearly exercise of ranking my favorite films a lot easier.

- What does it mean to take ratings seriously?
It's not an exact science, it's just a way to compartmentalize films. As long as films within the same rating point more or less belong together, I'm fine with it. I don't get people who sometimes "like a film better than another film, even though they gave it a lower rating". That's just nonsense to me.

- What is your rating scale of choice? (Thumbs, Stars, Die, 10, %)
I use stars, but only because MovieMeter uses stars. I'm fine with a scale /10 too, it's exactly the same thing for me.

- Do you think ratings communicate anything of value?
They do, but it does require you to know the rater a little bit. Once that understanding is there, it's very helpful in trying to find films you may have missed.

- If you see someone rate a film, say, 7 out of 10 - what information are you taking away from it.
Good but not great/not a personal favorite. But definitely worth watching.

- Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system?
It's just positive/negative. I don't mind the rating system itself, but the rotten/fresh thing is absolutely ridiculous. A film that only got 6s can still be 100% fresh, that makes no sense to me. I wish other sources would stop quoting their averages.

- Stars vs. 10/10 (Letterboxd vs. IMDb)
Both, no difference for me.

- What about Die? (Norway, Sweden)
Guess I'll have to listen to the episode for this. Die Bart Die?

- Is 1-100 or the percentage scale just too extreme?
It is for me, but to each his own. Not all brains work the same, if 1-100 is your thing, all the more power to you. I couldn't do it though.

- What does the lowest rating mean to you - what does the max rating represent? Is max always "perfect"? Is the minimum the opposite of perfect? And then I guess the question is - what fills up each of the steps in-between 1-10, 1-5, 1-100, etc.? What does each rating mean to you?
I kinda count down, so the minimum rating is for all the leftovers. The maximum rating certainly isn't for perfection, otherwise that would be a pointless rating for me personally.

4.0-5.0 is for personal favorites, from great to mindblowing
2.5-3.5 is for films I liked, from okay filler to prime filler
1.5-2.0 is for very basic stuff
0.5-1.0 is for crap, possibly with some redeeming element

I will say that my blog has influenced my ratings. The first year of my blog I kinda reviewed the films I liked to write about, without a real system in place. To make things a bit clearer (also for myself), I started reviewing all my 4.0* ratings and up. The funny thing is that actually impacted my ratings for the borderline cases (3.5-4.0). Since giving a 4* rating to a film now represented a real effort (writing the review), I got a bit stingier with my 4* ratings. It's funny how that works :)
User avatar
Teproc
Posts: 1171
Joined: September 23rd, 2015, 6:00 am
Contact:

#6

Post by Teproc »

Since I didn't participate in this one, let's do this:

- Do you rate films, and if so - for how long have you been doing it - and do you take ratings seriously?

I do, have one so ever since I seriously got into cinema, aka 2015 or thereabouts, and I suppose I do take it seriously in that I do refer back to my ratings for polls and such, and I do think of them as being meaningful and representative of how I feel about a film, though obviously nothing replaces, you know, words.

- What does it mean to take ratings seriously?

I guess I started answering that already. Maybe the big sign is when you start to worry about how your curve looks. I initially hoped for 6 to be my most common rating, but it turns out it's 7, which does annoy me a little bit. I also give out more 8s than I thought I would. I tell myself it's because I'm good at selecting stuff I'm likely to enjoy, but part of me also feels like I'm a tad generous at times.

- What is your rating scale of choice? (Thumbs, Stars, Die, 10, %)

Started out with LB stars and a x/10 for everywhere else, since these are really just the same scale. Once I started using Criticker, I did convert it all to a x/100 scale, which I still use in my personal stats and on that site, though i generally refer to my x/10 rating everywhere else, so as not to seem like (as much of) a psycho.

- Do you think ratings communicate anything of value?

Sure. They're a shorthand of how much I enjoyed a film. I do think they're primarily useful for myself (especially the x/100 ones), but they communicate a very quick, basic idea of how I feel about a film.

- If you see someone rate a film, say, 7 out of 10 - what information are you taking away from it.

They liked it. If they're Mike D'Angelo or Onderhond, that is quite significant. If they're an average LB user, it's probably a backhanded compliment. For an average ICMF user, I take it more in the way that I mean it myself, which is to say "Good movie".

- Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system?

I get why critics (you guys mentioned Ebert & Siskel in the podcast) would use it. It's basically "would I generally recommend this film or not" or "is this worth seeing yes or no", which is useful information, and often what it gets boiled down when you actually talk about films with other people in real life, if you think about it. I don't love the way Rotten Tomatoes presents it with the percentages though, for all the reasons that have been discussed on the podcast.

- Stars vs. 10/10 (Letterboxd vs. IMDb)

I find stars slightly more... I don't know, expressive? But they're basically the same thing.

- What about Die? (Norway, Sweden)

Always fun to hear about those crazy randomness-loving Nordics.

- Is 1-100 or the percentage scale just too extreme?

Well, I use it so I obviously don't think so. Again, I think it's more useful for organizational purposes than it is for communicating with other people, because "I rate this a 65" is so much harder to process than "I rate this 3 and a half stars or 7/10".

- What does the lowest rating mean to you - what does the max rating represent? Is max always "perfect"? Is the minimum the opposite of perfect? And then I guess the question is - what fills up each of the steps in-between 1-10, 1-5, 1-100, etc.? What does each rating mean to you?

Terrible to the point that I find it offensive. Either for moral/political reasons or for aesthetic reasons. I have absolutely no respect for this film whatsoever, no aspect of it is salvageable. I have only rated 7 films (out of 2304 atm) that rating of half a star or 1/10. As for my 100 scale, I have not yet rated anything beyond 6, but there's always hope !

Max rating means that the film is a part of me. It is not about "perfection" because there is no such thing, and even if there was I wouldn't be the one to declare it, it's just films that feel like they are mine in some way. I've got 32 5 stars ratings, and 5 of them are 100 on that scale.

5 stars - 10/10 - 91-100 : see above
4.5 stars - 9/10 - 81-90 : Films I love. Hard to define what separates them from 8s tbh, it's just a feeling, again maybe of something more personal.
4 stars - 8/10 - 71-80 : Films I like a lot. My threshold for films I'll vote for in ICM polls.
3.5 stars - 7/10 - 61-70 : Films I like.
3 stars - 6/10 - 51-60 : Films I think are fine. In French, I'd say "pas mal", which means "not bad" but I think is a bit more positive than it would be in English.
2.5 stars - 5/10 - 41-50 : Films I'm mixed on. It's the most mysterious of ratings, filled with stuff that's just kind of meh but with some redeeming qualities, or entertaining enough but that doesn't quite reach a posivite rating, or stuff that's all over the place and I don't know what to do with.
2 stars - 4/10 - 31-40 : Films I don't like, but that I don't dislike. Often films where I recognize the technical proficiency or the artistic intent, but just don't work for me.
1.5 stars - 3/10 - 21-30 : Films I dislike but with a notable redeeming quality.
1 star - 2/10 - 11-20 : Films I dislike.
0.5 star - 1/10 - 1-10 : See above. I guess you could say fllms I hate.

As a side note, when I see a film, I mark it/rate it on a whole 6 websites. ICM, LB, IMDB, Crticker and then two French sites, Allociné and Senscritique, basically the French equivalents to IMDB and Letterboxd, and both use a 5-star system. Allociné is perhaps the only website where I use the average ratings a lot (I never really look at it on LB or IMDB), specifically for new releases in French cinemas; if there's a film I know nothing about but has a press rating of 3.5 or above, I generally make an effort to see it. Don't pay much attention to the user rating average though.
User avatar
Onderhond
Posts: 7519
Joined: December 23rd, 2012, 7:00 am
Contact:

#7

Post by Onderhond »

Teproc wrote: September 1st, 2021, 9:50 pm They liked it. If they're Mike D'Angelo or Onderhond, that is quite significant.
Believe it or not, but 3.5 is my most popular rating for a film. As my curve shows (I don't have all my ratings there anymore, but it's close enough), it does drop off much faster on the positive side compared to the negative side.

I do understand the perception though, since a lot of my negatives overlap with the primary focus of ICM(f).
User avatar
Teproc
Posts: 1171
Joined: September 23rd, 2015, 6:00 am
Contact:

#8

Post by Teproc »

Yeah, I guess it's also because I pay more attention to ratings of films I've seen/heard of, which also contributes to you seeming to me like a tougher grader than you actually are.
blocho
Donator
Posts: 4945
Joined: July 20th, 2014, 6:00 am
Contact:

#9

Post by blocho »

Do you rate films, and if so - for how long have you been doing it - and do you take ratings seriously?
I do not rate movies. I very occasionally rank movies for forum polls and discussions, but I mostly avoid this as well. At times over the years, I've had cause to regret my refusal to rate movies. Certainly, it would have been helpful while trying to think about movies I don't remember very well, but there's a better solution to this problem (write a few sentences about the movie rather than just give it a rating).

This is something I've written about previously on the forum, but I'll restate it here. Over the past few years, I've tried to move my thinking away from the good/bad paradigm and toward an interesting/uninteresting paradigm. Ratings could be used for either, of course, but on the whole I find that ratings do not encapsulate what I want to take away from a movie and often lead instead to the sort of internal ranking and debate ("Is this movie really better than that movie?") that I don't want.

Do you think ratings communicate anything of value?
They can. In the aggregate, especially, if you believe in the whole wisdom of crowds theory. Take a movie that has a 7.5 on imdb. I might like it or I might hate it. But if I watched 100 movies with a 7.5 on imdb and 100 movies with a 5.5 on imdb, I know for sure that I'd like a lot more of the 7.5 movies than the 5.5 movies. The same goes for Rotten Tomatoes. I recognize that both of these sites don't quite fulfill the wisdom of crowds ideal -- they are not perfect aggregations of independent opinion. But they can be useful.

If you see someone rate a film, say, 7 out of 10 - what information are you taking away from it.
Very little unless the person doing the rating is someone who has very similar tastes to my own. There are only two or three people on the forum who would qualify, and even in their case, I've definitely disliked movies that they've loved.

Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system?
I'm with Teproc. I get why it's useful for critics, but I don't like it overall. With regard to Rotten Tomatoes, for example, it completely ignores the reality that there are many critics who don't give the movies they see a simple good/bad designation.

What about Die? (Norway, Sweden)
Like James Bond, I have No Time to Die.

Is 1-100 or the percentage scale just too extreme?
Not extreme enough. Back when I was young, my friends and I used the 10-200 scale to rate the attractiveness of women (please excuse the boorishness of my teenage self). Like the Richter Scale, it was logarithmic, and none of us understood exactly how it worked. I'm sure that moving to a similar scale for movie ratings would benefit everyone here.

What does the lowest rating mean to you - what does the max rating represent?
I notice that the majority of people here who rate movies often avoid the lower ratings entirely. Aside from a very few people, it's rare to see a rating below 4/10. In part, I think those ratings are due to some self-selection bias (the people in this forum tend to be people who like movies and thus are likely to give movies good ratings). But it's also just grade inflation.
Last edited by blocho on September 6th, 2021, 9:45 pm, edited 2 times in total.
User avatar
OldAle1
Donator
Posts: 6461
Joined: February 9th, 2017, 7:00 am
Location: Dairyland, USA
Contact:

#10

Post by OldAle1 »

I like blocho's post. I really wish I could re-start and not rate films, and remove all my ratings, but I guess after almost 40 years of doing it, I'm too attached to it.

I listened to the podcast, shortly after my own post, just forgot to come back to this thread until now. Wish it was fresher in my mind but I did want to comment on the IMDb ratings discussion - I'll say for myself that while I don't trust IMDb as a whole, I trust certain areas more than others, e.g. most classic-era noir and westerns seem to have "appropriate" ratings to me. It's mostly the post-1990 or so era that's problematic, both because of heavy trolling and because I tend to agree less with the typical IMDb user, who at this point is probably a few years younger than me and much more mainstream-oriented. But I think this brings up something valuable for anybody who is looking at ratings - of anything, on any site or from any source - which is parsing the ratings, and doing your homework. My brother apparently spends inordinate amounts of time reading Amazon reviews for stuff - because he buys a lot there, and doesn't want to get any lemons - and he has a whole convoluted system worked out that apparently does it for him, just as I suppose my system for using IMDb ratings works for me. For the most part; trolls and bots get more sophisticated all the time, which makes the work of ignoring them or adjusting for them a little harder all the time.

Anyway great discussion guys, as usual.
User avatar
mightysparks
Site Admin
Posts: 31618
Joined: May 5th, 2011, 6:00 am
Location: Perth, WA, Australia
Contact:

#11

Post by mightysparks »

Do you rate films, and if so - for how long have you been doing it - and do you take ratings seriously?

I have been rating films since around 2004 on IMDb and I take them somewhat seriously. I find them fun.

What does it mean to take ratings seriously?

For me it's because they make it easy for me to find films I liked/didn't like for polls or rewatches or comparisons etc. I agree with Onderhond that it's never an exact science but I am one of those people who "like a film better than another film, even though they gave it a lower rating". This is usually because I've given a film an 8 when I was 15 but it has no freshness or 'memory' for me anymore but a film I've seen more recently with a lower rating is still fresh and impactful in my mind.

What is your rating scale of choice? (Thumbs, Stars, Die, 10, %)

Out of 10. Mostly because I started out rating on IMDb. My new rating system does have a percentage too.

Do you think ratings communicate anything of value?

As some mentioned, they are useful as shorthand. My own ratings are useful for myself, especially when filtering in a spreadsheet or IMDb when making polls. I do pay attention to other people's ratings but I find them only useful if that person's rating language is similar to my own. People who don't rate below or above certain numbers, for example, are meaningless to me.

If you see someone rate a film, say, 7 out of 10 - what information are you taking away from it.

For me a 7 could be a favourite so a 7 will generally catch my eye as something to be noticed. But I've noticed that a lot of people don't seem to consider 7 a good rating--I've seen 3.5/5 reviews on LB saying 'this film didn't really do a lot for me' and it's like wtf :/ A 7/10 at uni would be a distinction, the second best grade you can get, so it's a meaningful rating to me.

Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system?

I hate it. A pointless system. Gives me no information.

Stars vs. 10/10 (Letterboxd vs. IMDb)

I prefer 10/10 but stars are fine too. I don't like anything that doesn't go into 10 or let me do half ratings though (Goodreads is super frustrating as it's out of 5 and you can't give half marks).

What about Die? (Norway, Sweden)

Haven't really come across this but it would probably annoy me.

Is 1-100 or the percentage scale just too extreme?

Like some others, I think this is useful for organisation. 5/10 just means 50% so it doesn't really seem that extreme. My new rating system uses percentage and that just helps me classify films rather than have all x's mean the same thing but it's not always helpful.

What does the lowest rating mean to you - what does the max rating represent? Is max always "perfect"? Is the minimum the opposite of perfect? And then I guess the question is - what fills up each of the steps in-between 1-10, 1-5, 1-100, etc.? What does each rating mean to you?

My ratings are generally about engagement/interest/impact/value, or the experience of the film. My ratings are always completely personal. Max does not mean perfect in a traditional sense but more that it provided me with a mindblowing experience (in whatever way it did that). The lowest rating would be one that is the opposite I suppose, a worthless, painful and 'unengaging' experience. I also don't really compare films when I rate them, eg 'I gave x a 7 and I liked this one more so I have to give it an 8' or whatever.

My new system gives 25% to story/performance/visuals/editing/atmosphere (not how 'good' they are, but moreso what they provided to the film experience) and 75% to creativity/engagement/impact.

5 stars - 10/10 - 95-100 : masterpiece/all-time favourite/seen a million times and still get the same incredible experience. engaging and amazing in all/almost all aspects
4.5 stars - 9/10 - 85-94 : 'perfect'/outstanding/brilliant experience that just lacks that 'oomph' or I simply have not seen it enough times
4 stars - 8/10 - 75-84 : favourite threshold and the highest rating I will give a film I've only seen once. One that is very impactful and engaging.
3.5 stars - 7/10 - 65-74 : A very good film, solid in most elements but not massively impactful or engaging.
3 stars - 6/10 - 55-64 : A good film that had little impact and little engagement. Could also be decently engaging with little impact.
2.5 stars - 5/10 - 45-54 : Mediocre experience that may have some ok elements but failed to be engaging or impactful. My most common rating.
2 stars - 4/10 - 35-44 : Bad/poor experience, probably bad in all elements and a major failure at engagement.
1.5 stars - 3/10 - 25-34 : Terrible/trash. My dislike threshold. These are films that make me angry they wasted my time and caused stomach pain from their terribleness. Possibly some 'ok' objective elements but something that failed in every other way.
1 star - 2/10 - 15-24 : Awful. Very minor redeeming factors.
0.5 star - 1/10 - 1-14 : Absymal. No redeeming factors.
"I do not always know what I want, but I do know what I don't want." - Stanley Kubrick

iCM | IMDb | Letterboxd | LastFM | TSZDT

Image
User avatar
Pretentious Hipster
Donator
Posts: 21543
Joined: October 24th, 2011, 6:00 am
Contact:

#12

Post by Pretentious Hipster »

Although I deal with this with games now, I find that the fundamental flaw of scores it that people just focus on that, even if you've done an entire review of it. Now I just say "I enjoy it for these reasons, and you might enjoy it if you enjoy these kinds of things"
User avatar
sol
Donator
Posts: 12388
Joined: February 3rd, 2017, 7:00 am
Location: Perth, WA, Australia
Contact:

#13

Post by sol »

Pretentious Hipster wrote: September 6th, 2021, 11:21 pm Although I deal with this with games now, I find that the fundamental flaw of scores it that people just focus on that, even if you've done an entire review of it.
I agree and I can't remember if I mentioned in the podcast or not, but yeah this happens, and certainly on Letterboxd where random reviewers will instantly like 50 or more reviews of similar rating just in order to try to get the likes back.
|iCM | IMDb | Letterboxd | Gold Derby
Image
Long live the new flesh!
User avatar
St. Gloede
Moderator
Posts: 12709
Joined: May 6th, 2011, 6:00 am
Contact:

#14

Post by St. Gloede »

Teproc wrote: September 1st, 2021, 9:50 pm - Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system?

I get why critics (you guys mentioned Ebert & Siskel in the podcast) would use it. It's basically "would I generally recommend this film or not" or "is this worth seeing yes or no", which is useful information, and often what it gets boiled down when you actually talk about films with other people in real life, if you think about it. I don't love the way Rotten Tomatoes presents it with the percentages though, for all the reasons that have been discussed on the podcast.
That's a great point, should you or should you not see this film is a fairly valuable set of information. I guess my issue with thumbs up and thumbs down, especially for RT (I remember Ebert giving Full Metal Jacket thumbs down despite thinking it a solid film, so his standards were higher and more relative).

Would I really tell people that they should go and see a film that's "not bad", and yes, that is the term we often use in Norwegian too, to someone else. There are so many great movies out there that even recommending really good movies, unless they have something extra interesting about them, may not seem that important.

Obviously, there may even be films that does not even work for me personally that I could for some reason recommend, for instance if I know that style tends to work for many others - but if it is a, say, perfectly ok drama with solid performances, I generally don't even try to seek them out myself, let alone throw them on others - hence my particular dislike of the system.


- Stars vs. 10/10 (Letterboxd vs. IMDb)

I find stars slightly more... I don't know, expressive? But they're basically the same thing.
As I said in the podcast I prefer the 10/10 scale, and with decimals, making it a 20 point scale, but that, I suppose is mainly relevant for me.

I will say however that in terms of actual communication the simple 5 star point may be even better due to its simplicity - which it slso shares with the due-system - which does not allow decimals.
- What about Die? (Norway, Sweden)

Always fun to hear about those crazy randomness-loving Nordics.
(l)

Oh, and the usual scale is:

6 = Masterpiece
5 = Great
4 = Good
3 = Ok/Mediocre
2 = Bad/Poor
1 = Terrible
- Is 1-100 or the percentage scale just too extreme?

Well, I use it so I obviously don't think so. Again, I think it's more useful for organizational purposes than it is for communicating with other people, because "I rate this a 65" is so much harder to process than "I rate this 3 and a half stars or 7/10".
Yeah, that is the main reason I used it to. Made rankings in particular much easier.
User avatar
St. Gloede
Moderator
Posts: 12709
Joined: May 6th, 2011, 6:00 am
Contact:

#15

Post by St. Gloede »

Onderhond wrote: September 1st, 2021, 9:13 pm - Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system?
It's just positive/negative. I don't mind the rating system itself, but the rotten/fresh thing is absolutely ridiculous. A film that only got 6s can still be 100% fresh, that makes no sense to me. I wish other sources would stop quoting their averages.
Amazing, exactly what I said and feel as well. :cheers:
User avatar
kongs_speech
Posts: 2078
Joined: April 4th, 2020, 10:32 pm
Location: FL
Contact:

#16

Post by kongs_speech »

Do you rate films, and if so - for how long have you been doing it - and do you take ratings seriously? I joined Letterboxd in 2018 and attempted to rate every film I've ever seen, at least the ones that I remember well enough to rate. Yes, I take them very seriously.

What does it mean to take ratings seriously? I'd say it means that one puts some thought into their scores and attempts to maintain some sort of consistency.

What is your rating scale of choice? (Thumbs, Stars, Die, 10, %) Stars from 0.5 to 5, because of Letterboxd. I have recently begun using a nine category system, with 10 possible points each in the categories of Creativity, Direction, Writing, Acting, Cinematography, Production Value, Pacing, Structure and Sound/Music. I divide the total by 90, multiply by 10 and use that number to determine my star rating. I have a spreadsheet for all of this, and I'm about 1/3 of the way through re-rating every film I've seen. As I go, I'm keeping a list of films that need to be rewatched before I can fairly rate them by category.

Do you think ratings communicate anything of value? Yes, it briefly and concisely tells what someone thinks of a film.

If you see someone rate a film, say, 7 out of 10 - what information are you taking away from it. It depends on the person. For me, 3.5/5 means "good," with 3/5 being mediocre and anything else below average.

Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system? No, I think the pass/fail system of Rotten Tomatoes is rather asinine, especially now that it has been watered down to include every nobody with a blog.

Stars vs. 10/10 (Letterboxd vs. IMDb) Stars all day.

What about Die? (Norway, Sweden) I'll play the stupid American card and admit I don't know what that means.

Is 1-100 or the percentage scale just too extreme? Not if you have a categorical rating system like I do. If you're just pulling a number out of your ass like "hmmm, Solaris strikes me as a 73/100," then I call shenanigans.

What does the lowest rating mean to you - what does the max rating represent? Is max always "perfect"? Is the minimum the opposite of perfect? And then I guess the question is - what fills up each of the steps in-between 1-10, 1-5, 1-100, etc.? What does each rating mean to you? A 0.5, especially under my current system, is absolute bottom of the barrel dreck, whereas anything that earns a 5 (9.7/10 or higher average) is practically perfect. I do suppose that makes them the opposite.

5/5 - masterpiece
4.5/5 - near-masterpiece
4/5 - great
3.5/5 - good
3/5 - average
2.5/5 - bad
2/5 - very bad
1.5/5 - painful
1/5 - one of the worst of its year
0.5/5 - one of the worst ever made
🏳️‍⚧️
Quartoxuma wrote: A deeply human, life-affirming disgusting check whore.
Image
User avatar
St. Gloede
Moderator
Posts: 12709
Joined: May 6th, 2011, 6:00 am
Contact:

#17

Post by St. Gloede »

blocho wrote: September 2nd, 2021, 2:53 am Do you rate films, and if so - for how long have you been doing it - and do you take ratings seriously?
I do not rate movies. I very occasionally rank movies for forum polls and discussions, but I mostly avoid this as well. At times over the years, I've had cause to regret my refusal to rate movies. Certainly, it would have been helpful while trying to think about movies I don't remember very well, but there's a better solution to this problem (write a few sentences about the movie rather than just give it a rating).

This is something I've written about previously on the forum, but I'll restate it here. Over the past few years, I've tried to move my thinking away from the good/bad paradigm and toward an interesting/uninteresting paradigm. Ratings could be used for either, of course, but on the whole I find that ratings do not encapsulate what I want to take away from a movie and often lead instead to the sort of internal ranking and debate ("Is this movie really better than that movie?") that I don't want.
Yes, that is a very good way of thinking about films and even films that don't work for you can still lead to incredible discussions and great insights.

That said, and not to rub salt in your wounds, yes, not eating films or keeping a record of how you felt will make remembering films much, much harder and truly work against you if you at some point seek to fully join us crazy full on list freaks.
Do you think ratings communicate anything of value?
They can. In the aggregate, especially, if you believe in the whole wisdom of crowds theory. Take a movie that has a 7.5 on imdb. I might like it or I might hate it. But if I watched 100 movies with a 7.5 on imdb and 100 movies with a 5.5 on imdb, I know for sure that I'd like a lot more of the 7.5 movies than the 5.5 movies. The same goes for Rotten Tomatoes. I recognize that both of these sites don't quite fulfill the wisdom of crowds ideal -- they are not perfect aggregations of independent opinion. But they can be useful.
Excellent point and one I wish we had investigated in the episode in more depth. Somehow we ended up discussing how the aggregate influences people to see films, how they can be manipulated and how we can read through the aggregate to find low performing films we like more than how strong the IMDb and Letterboxd aggregate actually are...
What about Die? (Norway, Sweden)
Like James Bond, I have No Time to Die.
:poshclap:
What does the lowest rating mean to you - what does the max rating represent?
I notice that the majority of people here who rate movies often avoid the lower ratings entirely. Aside from a very few people, it's rare to see a rating below 4/10. In part, I think those ratings are due to some self-selection bias (the people in this forum tend to be people who like movies and thus are likely to give movies good ratings). But it's also just grade inflation.
I think it is more that we are more selective here, thanks to using list sites like ICM, aggregates like IMDb and Letterboxd and of course critics and recommendations. It is very hard to watch bad films when you do this, unless you have tastes that generally don't match up with most other film buffs. In general, at least for me, it is nearly impossible to see broadly critically acclaimed films that are actually bad - 4 (poor) being about as low as it can get, and even that almost never happens as these films will, at the very least, have the basics down and do something of note. The exceptions, for me, come when we enter fringe niches with only partial critical support, but here too there is usually "something" for me there.

Back before I followed ratings, critics, etc. though, the majority of films I saw were much, much weaker, and it is very easy to find the bad films if I want to - but also very, very easy to avoid them.
User avatar
St. Gloede
Moderator
Posts: 12709
Joined: May 6th, 2011, 6:00 am
Contact:

#18

Post by St. Gloede »

OldAle1 wrote: September 6th, 2021, 8:49 pm I like blocho's post. I really wish I could re-start and not rate films, and remove all my ratings, but I guess after almost 40 years of doing it, I'm too attached to it.

I listened to the podcast, shortly after my own post, just forgot to come back to this thread until now. Wish it was fresher in my mind but I did want to comment on the IMDb ratings discussion - I'll say for myself that while I don't trust IMDb as a whole, I trust certain areas more than others, e.g. most classic-era noir and westerns seem to have "appropriate" ratings to me. It's mostly the post-1990 or so era that's problematic, both because of heavy trolling and because I tend to agree less with the typical IMDb user, who at this point is probably a few years younger than me and much more mainstream-oriented. But I think this brings up something valuable for anybody who is looking at ratings - of anything, on any site or from any source - which is parsing the ratings, and doing your homework. My brother apparently spends inordinate amounts of time reading Amazon reviews for stuff - because he buys a lot there, and doesn't want to get any lemons - and he has a whole convoluted system worked out that apparently does it for him, just as I suppose my system for using IMDb ratings works for me. For the most part; trolls and bots get more sophisticated all the time, which makes the work of ignoring them or adjusting for them a little harder all the time.

Anyway great discussion guys, as usual.
Noooo, don't go to the statless side, that way lies chaos.

:D

Anyways, thank you so much, and yes, that is definitely the best way of using IMDb. I have undoubtedly gotten less and less interested in the ratings and use them only as a pointer - putting more and more weight on exciting reviews, recommendations and frankly, our own internal polls and lists (and other reason to not abandon stata ;) )
matthewscott8
Donator
Posts: 2218
Joined: May 13th, 2015, 6:00 am
Contact:

#19

Post by matthewscott8 »

Thumbs up or down has a bad association I guess, it's reminiscent of the classical image of the emperor in a coliseum deciding whether a gladiator gets dispatched or not. The critic is not the emperor so I would not use thumbs up and down. The best critics go onto make great films themselves, sort of a promotion. Some level of humility and symbiosis with the system is necessary from the critic.

I think that I can possibly get something out of a star system even if I don't agree with the critic's assessment. It works something like this. I often look for words like ambitious and atmospheric in movie reviews. Those tend to mean I will like the movie even if the critic didn't. So if you had several categories which you rated 1-5 then that could work. Like in a synchro swimming or gymnastics competition you get scored on ambition, artistic content and execution.

Outside of a more complex rating like that, I would mostly only see the star system as relevant if I knew I had some sort of alignment with the author.

Little White Lies have a system of 3 ratings out of 5, for anticipation, enjoyment and "in retrospect". A datum worth throwing into the mix if no-one else has mentioned. These ratings tend not to differ much though.
User avatar
Reflect
Posts: 521
Joined: February 6th, 2017, 7:00 am
Contact:

#20

Post by Reflect »

I was actually just thinking about this subject!

- Do you rate films, and if so - for how long have you been doing it - and do you take ratings seriously?

I've been rating films ever since I became seriously passionate about them as a kid and started "logging" them on IMDb. I take ratings seriously in the sense that I put real thought into what I'd rate something, because to me, it's a broad assessment of how much I liked/appreciated the film. As far as other people's ratings, it sort of depends. There are some whose taste and writing I know and respect enough that their ratings do mean a good deal to me on the basis of having a rough idea of where they stand on something. Then there are some people who actually are just trying to be provocative with their ratings, and the obvious disingenuous slant to that makes it mean next to nothing to me.

- What does it mean to take ratings seriously?


To value what they stand for, I guess.

- What is your rating scale of choice? (Thumbs, Stars, Die, 10, %)

1-10, which is what I used on IMDb and now Letterboxd (it's a five star system, but there are half stars, so it amounts to 10).

- Do you think ratings communicate anything of value?

At times, yes. At other times, literally nothing.

- If you see someone rate a film, say, 7 out of 10 - what information are you taking away from it.

Depends on the person. For me, a 7 broadly means "I liked but didn't love this".

- Thumbs up/Down (Fresh/Rotten - RottenTomatoes) - What is up, what is down - and is this a remotely good system?


I've never cared about Rotten Tomatoes. The system is extremely broad, I prefer not being THAT general, but eh.

- Stars vs. 10/10 (Letterboxd vs. IMDb)

Same thing for me.

- What about Die? (Norway, Sweden)


Don't know it, will have to listen to the podcast.

- Is 1-100 or the percentage scale just too extreme?

Yeah, I don't see anyone really being able to explain there being any meaningful difference between a 61/100 and a 62/100. But to each their own.

- What does the lowest rating mean to you - what does the max rating represent? Is max always "perfect"? Is the minimum the opposite of perfect? And then I guess the question is - what fills up each of the steps in-between 1-10, 1-5, 1-100, etc.? What does each rating mean to you?

The lowest simply means the films I loved/appreciated the least, and the max means the films I loved/appreciated the most. The "perfect" qualifier has always seemed silly to me. I don't think art is there to be "perfect", but that could also mean a lot of things depending how you frame it.
User avatar
Torgo
Posts: 3906
Joined: June 30th, 2011, 6:00 am
Location: Germany
Contact:

#21

Post by Torgo »

👍🏽

I have a long answer for this where I could go into each of the questions posed in the OP, but for now I want to narrow the discussion down to the most important argument for ratings:

Do I have time for (or am interested in) reading every single review of every single poster in the forum? Absolutely fucking not. And this is not meant offensive against our users who decide against ratings - some of my best friends favorite users don't use ratings! ;) - but one just has to filter the endless stream of text and opinion on the net. Yes, a good review text by someone who I can relate to or even just learn something from is more worth than a 7/10 from a random or somebody whose opinion doesn't mean a lot to me. Still I must be able to scroll through a thread (or a list, or a blog post,..) and weed out the filler stuff to make a pause when someone posts a 9/10 or 10/10 and let my brain do the work: "Oh, an exceptionally good score - this might be worth to memorize. / .. Oh, I've heard of this one before! Maybe I should finally give it a watch. / .. Oh, this seems to polarize a lot, didn't the other guy rate just this a 4/10 before? Peculiar. / .. 0/5 for THIS? User xy just earned a trip to my ignore list!!"

And this is true for a huge portion of movies, because I only know a small pond of films while the forum (and the web) watches and writes about the complete ocean. In the end, a film title is just a sequence of letters. If I don't know it or don't know the director (and don't have a cover/poster or more information ready), it could be just everything. Can't read about just everything all the time.
Do I want to read through every summary of newly released films each week on Metacritic or even go further and read a collection of reviews everytime? Or will it suit me just watching out for good / very good ratings, ignore the mixed ones for a while and chuckle about red scores? "Cool, Green Knight has one of the highest scores of the season - I will prioritize this. Old and Reminiscence don't look terribly special. Prio tier C for now if I don't hear something else. Space Jam didn't get a single good rating anywhere, neither IMDb or Metascore or word of mouth - doesn't seem to be a new Lego Movie hit. Pushed back waayy at the end of my 2021 queue."

👎🏽
User avatar
mightysparks
Site Admin
Posts: 31618
Joined: May 5th, 2011, 6:00 am
Location: Perth, WA, Australia
Contact:

#22

Post by mightysparks »

That’s a good point Torgo. I don’t like reading reviews of films I haven’t seen so I tend to look for the high or low ratings (and skim for keywords) to decide whether I want to check something out or actively avoid it.
"I do not always know what I want, but I do know what I don't want." - Stanley Kubrick

iCM | IMDb | Letterboxd | LastFM | TSZDT

Image
Post Reply