Welcome to the ICM Forum. If you have an account but have trouble logging in, or have other questions, see THIS THREAD.
Podcast: Talking Images (Episode 11 released July 24th)
Polls: 0 Official Lists (Results), 1960 (Aug 9th), Romance (Aug 28th)
Challenges: German/Austrian/Swiss, <400, 1970s
Film of the Week: Sibiriada, September nominations (Aug 28th)
World Cup S4: Match 2G: Russia vs Portugal (Aug 6th), Match 2H: India vs Cuba (Aug 16th), QF Preparation (Aug 25th)

Update IMDb-info on an iCM-page

Post Reply
User avatar
joachimt
Donator
Posts: 31923
Joined: Feb 16, 2012
Location: Netherlands
Contact:

Update IMDb-info on an iCM-page

#1

Post by joachimt » July 5th, 2020, 7:56 pm

Since yesterday everyone has the ability to update IMDb-info on iCM-pages. If you see a missing runtime, director, genre or AKA or something is wrong like a messed up title or name, you can update that info yourself. Note that you can't edit info, you can only make iCM extract the info from IMDb. For more info read this post.

How does it work? Go to the beta-page of a movie. Click the button with the three dots. Click "Update info".

If anything doesn't work like you would expect, feel free to ask here for help.

You can't fix duplicates yourself, so if you find any of those, you still need to report them. You can post them here.
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
Ebbywebby
Posts: 3360
Joined: Sep 10, 2012
Location: Orange County, CA
Contact:

#2

Post by Ebbywebby » July 5th, 2020, 9:03 pm

Runtimes and AKAs seem like the fields with the most gaps that can be filled in.

In the beta world, there are currently about 4172 pages of films, TV episodes, etc. without runtimes. Or about 208K individual items. Not so long ago, there were 4213 pages. I wonder how low this tally could go, if everyone makes an effort to fill in their own checks (or favorite unofficial lists) with missing runtimes? (But of course, new pages with empty fields get created all the time....)

Warning: There are many, many pre-1910 film snippets without runtimes, and they're rarely update-able. Futile.

User avatar
Ebbywebby
Posts: 3360
Joined: Sep 10, 2012
Location: Orange County, CA
Contact:

#3

Post by Ebbywebby » July 8th, 2020, 12:12 am

I just fixed the titles of what must have been 350-400 episodes of some anime series called "One Piece: Wan pîsu." Because of ICM's problem with accents, every episode was identically identified as just "One Piece: Wan p."

Another extra-good "accent error" that I recently fixed: four Victor Sjöstrom films that were credited to "Victor Sj."

User avatar
joachimt
Donator
Posts: 31923
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#4

Post by joachimt » July 8th, 2020, 5:43 pm

Great work, EW!
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
shugs
Donator
Posts: 568
Joined: Nov 15, 2014
Location: Bucharest, Romania
Contact:

#5

Post by shugs » July 9th, 2020, 10:59 am

Couldn't this be automated somehow? It seems like never-ending work.

TMDB has the changes API for recently changed data (https://developers.themoviedb.org/3/cha ... hange-list) , if ICM is ever going to use it as a data source, and IMDb has the datasets (https://www.imdb.com/interfaces/) which maybe could be used.

User avatar
joachimt
Donator
Posts: 31923
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#6

Post by joachimt » July 9th, 2020, 3:57 pm

shugs wrote:
July 9th, 2020, 10:59 am
Couldn't this be automated somehow? It seems like never-ending work.

TMDB has the changes API for recently changed data (https://developers.themoviedb.org/3/cha ... hange-list) , if ICM is ever going to use it as a data source, and IMDb has the datasets (https://www.imdb.com/interfaces/) which maybe could be used.
We've talked about this in the past. It would require running a program that regularly hits the button for every moviepage on iCM. Marijn said it would be too heavy on the database to do this for everything. Another option would be to do such a thing only for certain filters, like recent movies or movies with a certain amount of checks.
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
shugs
Donator
Posts: 568
Joined: Nov 15, 2014
Location: Bucharest, Romania
Contact:

#7

Post by shugs » July 9th, 2020, 4:17 pm

joachimt wrote:
July 9th, 2020, 3:57 pm
We've talked about this in the past. It would require running a program that regularly hits the button for every moviepage on iCM. Marijn said it would be too heavy on the database to do this for everything. Another option would be to do such a thing only for certain filters, like recent movies or movies with a certain amount of checks.
Yeah, that indeed sounds like overkill. Well, here's to hoping IMDb becomes more developer friendly in the future. :cheers:

User avatar
Ebbywebby
Posts: 3360
Joined: Sep 10, 2012
Location: Orange County, CA
Contact:

#8

Post by Ebbywebby » July 9th, 2020, 6:58 pm

joachimt wrote:
July 9th, 2020, 3:57 pm
We've talked about this in the past. It would require running a program that regularly hits the button for every moviepage on iCM. Marijn said it would be too heavy on the database to do this for everything.
Well, what about doing it JUST ONCE, rather than "regularly"? It wouldn't be a perfect, eternal solution, but it would correct thousands of glitches.

User avatar
joachimt
Donator
Posts: 31923
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#9

Post by joachimt » July 9th, 2020, 7:31 pm

Ebbywebby wrote:
July 9th, 2020, 6:58 pm
joachimt wrote:
July 9th, 2020, 3:57 pm
We've talked about this in the past. It would require running a program that regularly hits the button for every moviepage on iCM. Marijn said it would be too heavy on the database to do this for everything.
Well, what about doing it JUST ONCE, rather than "regularly"? It wouldn't be a perfect, eternal solution, but it would correct thousands of glitches.
I'm not sure if this is something that can be done easily. Maybe Marijn even need to built something for this. Not sure if it would actually work as well. Doing it once would still create a huge queue.

One thing that comes to mind is writing a script for this. I guess it wouldn't be hard for someone like mjf to write a script that let's you enter a list of beta-urls and it automatically hits the button on those pages one by one. There has to be some time between two hits so the queue won't get stuck. I'd have to ask Marijn if he thinks iCM database could handle this.
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
Ebbywebby
Posts: 3360
Joined: Sep 10, 2012
Location: Orange County, CA
Contact:

#10

Post by Ebbywebby » July 10th, 2020, 7:05 am

joachimt wrote:
July 9th, 2020, 7:31 pm
[I'm not sure if this is something that can be done easily. Maybe Marijn even need to built something for this. Not sure if it would actually work as well. Doing it once would still create a huge queue.

One thing that comes to mind is writing a script for this. I guess it wouldn't be hard for someone like mjf to write a script that let's you enter a list of beta-urls and it automatically hits the button on those pages one by one. There has to be some time between two hits so the queue won't get stuck. I'd have to ask Marijn if he thinks iCM database could handle this.
I guess I don't have much sense of how long it would take the site to do 780,000 automated updates. What if the site went down one night for a few hours for "maintenance"? Would that be long enough?

How does the site add new movies and TV episodes as they're announced/released? There must be some sort of periodic IMDb parse, right?

Peripheral question: Is www.themoviedb.org's data fully trustable? I keep adding runtimes to IMDb, and I labor to find a couple of independent sources before I submit a given time. But can the TMDb be used as a confident, one-stop resource?

User avatar
Onderhond
Posts: 4573
Joined: Dec 23, 2012
Contact:

#11

Post by Onderhond » July 10th, 2020, 8:06 am

I wouldn't put too much trust in the TMDb data. I regularly add films there myself (to import them into Letterboxd) and there is no verification process to speak of.

User avatar
joachimt
Donator
Posts: 31923
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#12

Post by joachimt » July 10th, 2020, 10:00 am

Ebbywebby wrote:
July 10th, 2020, 7:05 am
How does the site add new movies and TV episodes as they're announced/released? There must be some sort of periodic IMDb parse, right?
Just like any other title. Someone needs to add it.
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
Ebbywebby
Posts: 3360
Joined: Sep 10, 2012
Location: Orange County, CA
Contact:

#13

Post by Ebbywebby » July 10th, 2020, 10:39 am

joachimt wrote:
July 10th, 2020, 10:00 am
Ebbywebby wrote:
July 10th, 2020, 7:05 am
How does the site add new movies and TV episodes as they're announced/released? There must be some sort of periodic IMDb parse, right?
Just like any other title. Someone needs to add it.
Wow, I did not realize this.

User avatar
Ebbywebby
Posts: 3360
Joined: Sep 10, 2012
Location: Orange County, CA
Contact:

#14

Post by Ebbywebby » July 14th, 2020, 6:30 am

Ebbywebby wrote:
July 5th, 2020, 9:03 pm
In the beta world, there are currently about 4172 pages of films, TV episodes, etc. without runtimes. Or about 208K individual items. Not so long ago, there were 4213 pages. I wonder how low this tally could go, if everyone makes an effort to fill in their own checks (or favorite unofficial lists) with missing runtimes?
Only 4041 pages now! Go...team?

User avatar
Lonewolf2003
Donator
Posts: 9256
Joined: Dec 29, 2012
Contact:

#15

Post by Lonewolf2003 » July 14th, 2020, 4:31 pm

Ebbywebby wrote:
July 14th, 2020, 6:30 am
Ebbywebby wrote:
July 5th, 2020, 9:03 pm
In the beta world, there are currently about 4172 pages of films, TV episodes, etc. without runtimes. Or about 208K individual items. Not so long ago, there were 4213 pages. I wonder how low this tally could go, if everyone makes an effort to fill in their own checks (or favorite unofficial lists) with missing runtimes?
Only 4041 pages now! Go...team?
Do you have an overview list? Or do you search or sort for runtime on iCM?

User avatar
Ebbywebby
Posts: 3360
Joined: Sep 10, 2012
Location: Orange County, CA
Contact:

#16

Post by Ebbywebby » July 15th, 2020, 7:44 am

I'm not sure what LW means by "overview list." But the answer is probably no.

I just discovered something rather dismaying. I stumbled upon a March, 2017 forum post where I joked that there were "only" 190,189 items left in the database without runtimes.

Now, over three years later and after oodles and oodles and oodles of runtime updates, that 190,189 tally has "shrunk" to about...201,250?! Sigh.

User avatar
Lonewolf2003
Donator
Posts: 9256
Joined: Dec 29, 2012
Contact:

#17

Post by Lonewolf2003 » July 15th, 2020, 8:13 am

Ebbywebby wrote:
July 15th, 2020, 7:44 am
I'm not sure what LW means by "overview list." But the answer is probably no.

I just discovered something rather dismaying. I stumbled upon a March, 2017 forum post where I joked that there were "only" 190,189 items left in the database without runtimes.

Now, over three years later and after oodles and oodles and oodles of runtime updates, that 190,189 tally has "shrunk" to about...201,250?! Sigh.
I meant a list with all titles with missing runtimes. And I don’t mean an iCM list. Could also be a excel sheet f.e.

User avatar
Ebbywebby
Posts: 3360
Joined: Sep 10, 2012
Location: Orange County, CA
Contact:

#18

Post by Ebbywebby » August 1st, 2020, 9:54 pm

I'm not sure how to diagnose this, but it seems like there are a lot of silent-era movies that scarcely anyone has seen that IMDb lists as 50 minutes long. Feels like kind of a "whatever, it's probably about that long" fudge.

Random example: https://beta.icheckmovies.com/movies/16 ... eye+of+god

Or, look how many Henry King films are supposedly 50 minutes long.

https://beta.icheckmovies.com/movies?fi ... rt=runtime

I don't know where to mention this since it's not really a "bug"....

User avatar
joachimt
Donator
Posts: 31923
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#19

Post by joachimt » August 2nd, 2020, 8:23 am

Might be sloppy adding to IMDb indeed. Nothing we can do about it. Except if you want to find evidence on each one of them and change/remove the runtimes on IMDb that are incorrect.
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
Ebbywebby
Posts: 3360
Joined: Sep 10, 2012
Location: Orange County, CA
Contact:

#20

Post by Ebbywebby » August 2nd, 2020, 11:08 am

Just came across three more, practically in a row. Zero checks, 50 minutes.

https://beta.icheckmovies.com/movies/16 ... s+of+youth
https://beta.icheckmovies.com/movies/16 ... +rebellion
https://beta.icheckmovies.com/movies/17 ... e+fire+cat

3950 pages of no-runtime films left in beta ICM.

User avatar
Fergenaprido
Donator
Posts: 4215
Joined: Jun 03, 2014
Location: Malaysia
Contact:

#21

Post by Fergenaprido » August 3rd, 2020, 4:31 pm

The other possibility could be that runtimes were more rigid back in the early days because it was determined by the length of the film itself (1-reel, 2-reel, 3-reel, etc.) and they probably wanted to use up an entire reel so as not to waste their investment, so the fact that so many are of the relatively same length doesn't surprise me too much. I could also be completely wrong, though :D

Post Reply