Welcome to the ICM Forum. If you have an account but have trouble logging in, or have other questions, see THIS THREAD.
Podcast: Talking Images (Episode 16 released September 13th)
Polls: Romance (Results), 1951 (Results), 500<400 (Sep 23rd), 2008 (Oct 4th)
Challenges: Animation, Silent Era, Russia/USSR
Film of the Week: Durak, October nominations (Sep 25th)
World Cup S4: QF Schedule, Match QFB: India vs Greece (Sep 20th), Match QFC: Germany vs Italy (Oct 1st)

User-made scripts and apps etc

User avatar
Minkin
Posts: 540
Joined: Jan 13, 2015
Location: Ventura County, CA
Contact:

Re: User-made scripts and apps etc

#881

Post by Minkin » August 4th, 2020, 8:56 pm

@Max, that's rather beyond my capabilities. But that looks like a great solution! I can't be the only person interested in such an endeavor.

And thanks Fergenaprido. Perhaps when I'm looking for official films from a country, I'll just consult ICMforum after doing my preliminary research -as there should be somebody with an answer or some suggestions!

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#882

Post by Tim2460 » August 4th, 2020, 9:36 pm

maxwelldeux wrote:
July 31st, 2020, 11:46 pm
Minkin wrote:
July 31st, 2020, 8:48 pm
I was hoping somebody had a handy script for extracting country from IMDB and using that in combination with lists. But alas.

Looks like I'm well covered with Egypt (and every other country on the UNESCO + Asian list), but I'm going to start running into walls with certain countries - and just have to go with whatever Google turns up. It will just be annoying when I get to certain countries - like I'm sure there's some Croatian films on official lists, but good luck trying to figure out which (would start with Wikipedia and then cross-reference a Yugoslavia list).

Thanks for your help though + for pointing those lists out to me!
I looked into this, and it's totally possible. Get an API key from omdbapi.com ($1 will get you 100k calls/month, which is more than enough for official checks), use Tim's mega lists, get the tt-code, and call the API with the tt-code to get the country. Repeat for each official check.

I'd do it myself, but I don't have an overabundance of time right now (which is a good thing!).
Max : it seems the free omdbapi.com key is already 1000 request / day :party:

I'll take a look at that tomorow !

User avatar
maxwelldeux
Donator
Posts: 8169
Joined: Jun 07, 2016
Location: Seattle-ish, WA, USA
Contact:

#883

Post by maxwelldeux » August 4th, 2020, 9:49 pm

Tim2460 wrote:
August 4th, 2020, 9:36 pm
maxwelldeux wrote:
July 31st, 2020, 11:46 pm
Minkin wrote:
July 31st, 2020, 8:48 pm
I was hoping somebody had a handy script for extracting country from IMDB and using that in combination with lists. But alas.

Looks like I'm well covered with Egypt (and every other country on the UNESCO + Asian list), but I'm going to start running into walls with certain countries - and just have to go with whatever Google turns up. It will just be annoying when I get to certain countries - like I'm sure there's some Croatian films on official lists, but good luck trying to figure out which (would start with Wikipedia and then cross-reference a Yugoslavia list).

Thanks for your help though + for pointing those lists out to me!
I looked into this, and it's totally possible. Get an API key from omdbapi.com ($1 will get you 100k calls/month, which is more than enough for official checks), use Tim's mega lists, get the tt-code, and call the API with the tt-code to get the country. Repeat for each official check.

I'd do it myself, but I don't have an overabundance of time right now (which is a good thing!).
Max : it seems the free omdbapi.com key is already 1000 request / day :party:

I'll take a look at that tomorow !
1k/day is great, but when there are 15-20k official checks it'll take a couple weeks on the free plan. Which is totally fine, unless you want the full results tomorrow. :P

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#884

Post by Tim2460 » August 5th, 2020, 7:21 am

It looks promising ...
When i launch http://www.omdbapi.com/?i=tt3896198&apikey=xxxxxx
it gives me
Image

But that's an webpage... i can easily launch the 22000 firefox i need but I would need to save that info automatiquely. Any idea ?

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#885

Post by Melvelet » August 5th, 2020, 7:38 am

It's actually a json file (or rather a response that is structure like a json file that can also be parsed as such) that is rendered by your browser. Most programming languages are able to read it like a dictionary/similar file structure.
You can also download it from the browser (there's a button)

And if you want to save it automatically, you'll need to make an HTTP request (they'll possibly also have API implementations in the most common programming languages). Ic ould take a look later today.

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#886

Post by Tim2460 » August 5th, 2020, 9:33 am

Thx Melvelet :

I'll have an list of TTnumber to get info from. For ex :

tt0052357
tt0062622
tt0056172
tt0054215
tt0068646
tt0033467
tt0034583
tt0076759
tt0078788
tt0245429
tt0110912
tt0022100
tt0040522
tt0017136
tt0018455
tt0041959
tt0042876
tt0045152
tt0056801
tt0057012
etc etc...

I can easealy import the final json into excel as it seems. Idealy an JSON including all the 1000 film request of the day. Worst case scenario would be to have to include every JSON by hand one film at a time or copy paste the Country and all interesting collums.

Here's the file :
tt3896198.json.txt
You do not have the required permissions to view the files attached to this post.

User avatar
Onderhond
Posts: 4861
Joined: Dec 23, 2012
Contact:

#887

Post by Onderhond » August 5th, 2020, 9:56 am

Best to just write a script for it? JSON is made to be parsed programmatically, so running through all the imdb ids and parsing the feedback shouldn't be too hard.

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#888

Post by Melvelet » August 5th, 2020, 10:27 am

I'll write a python script. I just need to learn how to create an exe file from it. Should not be too hard

User avatar
Lonewolf2003
Donator
Posts: 9633
Joined: Dec 29, 2012
Contact:

#889

Post by Lonewolf2003 » August 5th, 2020, 11:37 am

maxwelldeux wrote:
August 4th, 2020, 9:49 pm
Tim2460 wrote:
August 4th, 2020, 9:36 pm
maxwelldeux wrote:
July 31st, 2020, 11:46 pm


I looked into this, and it's totally possible. Get an API key from omdbapi.com ($1 will get you 100k calls/month, which is more than enough for official checks), use Tim's mega lists, get the tt-code, and call the API with the tt-code to get the country. Repeat for each official check.

I'd do it myself, but I don't have an overabundance of time right now (which is a good thing!).
Max : it seems the free omdbapi.com key is already 1000 request / day :party:

I'll take a look at that tomorow !
1k/day is great, but when there are 15-20k official checks it'll take a couple weeks on the free plan. Which is totally fine, unless you want the full results tomorrow. :P
Maybe we can do it as a crowd? If multiple users of us do it a few days we could get more done quicker. If somebody writes a script that’s easy to use. I’m willing to help out.
Could be interesting information for polls and challenges too.

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#890

Post by Tim2460 » August 5th, 2020, 12:05 pm

Or i would need several keys : everyonecan get one for free on the website, i would then launch the script as pack of 1000 with each key.

Ok for the python script but i'm on a w10 machine as main rig.

My Nas / seeding machine is an Debian Linux Distro... it may be easier to lunch the script from that one ?
I could easily maintan an Excel File with all the extra collums of the json when i update the official lists every start of month.

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#891

Post by Melvelet » August 5th, 2020, 1:33 pm

Is it even so important to compile ten thousands of entries asap?
The tool would have an input file with tt-style imdb IDs and manage a permanent mapping file (either csv or json/yaml). For an input list, it would check if there's already a local entry for that id and make an API if there is none. So it's possible to have a long list of IDs and split it between multiple people and then to merge the resulting mapping together. Don't know if they have anything to detect API key abuse but creating lots of keys to avoid having a paid key seems to be the most obvious case of abuse :D

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#892

Post by Tim2460 » August 5th, 2020, 2:30 pm

We're clearly not on a Hurry.

Here's the first pack of 950 TTnumbers
You do not have the required permissions to view the files attached to this post.

User avatar
shugs
Donator
Posts: 587
Joined: Nov 15, 2014
Location: Bucharest, Romania
Contact:

#893

Post by shugs » August 5th, 2020, 2:34 pm

I could code something up quick in the weekend using the TMDb API (their data is not as exhaustive as IMDB's). I already have some scripts to get stats based on my views, cause I'm cheap and don't want to pay for Letterboxd Pro :sweat: I'd just need the full list of tt codes.

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#894

Post by Tim2460 » August 5th, 2020, 2:54 pm

We can't have enought data i guess ;) I do hope they have the country ?

Here's the full Official Pack if you wanna play with it a little :

https://we.tl/t-JqEnIGkqhG

User avatar
shugs
Donator
Posts: 587
Joined: Nov 15, 2014
Location: Bucharest, Romania
Contact:

#895

Post by shugs » August 5th, 2020, 3:02 pm

Yes, they have it. I use genres, countries, directors (name + gender), years and runtime values when playing with my stats. I'll see what I can generate during the weekend. Probably a huge CSV with everything and a MySQL dump for extra flexibility.

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#896

Post by Tim2460 » August 5th, 2020, 3:13 pm

shugs : :worship: :cheers: :poshclap:

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#897

Post by Melvelet » August 5th, 2020, 3:33 pm

Ah, the first version of the python tool would almost be ready :D

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#898

Post by Melvelet » August 5th, 2020, 4:43 pm

What exactly was the original usecase? Find all official movies from country x? I'll add a command line option to filter that.
Anyway, I attached the mappings that my script created until the API call limit was reached. I can easily extend the script to attach other OMDB info like Directors, Runtime etc.
The tool allows to add info to any input list of imdb urls/ids (both work) so it could also easily be used to add omdb info to Poll results, personal film log spreadsheets, etc. If anybody has any special requirements/ideas, please share
And we can keep an extensive "master list" in which we can manually clean "irrelevant" co-production countries etc.

If somebody is interested, I can share the GH repo with them, otherwise I'll post the script after I added some more functionality (like the country filter)

And I think I can create a Win executable file that does not require Python or any of the dependencies
You do not have the required permissions to view the files attached to this post.

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#899

Post by Tim2460 » August 5th, 2020, 7:55 pm

Melvelet :

That's exactly what we needeed in the first place.

But now let's go for the whole lot : having all the collums could save me a lot of time when building the main excel file to create the official lists every month.

For the master list, good idea : i can copy the official excel list to an google form.

Even better will be to make it grow and live of it's own, from the master list of all the screenshots we use to create the pool results.

Excellent job so far :banana:

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#900

Post by Tim2460 » August 6th, 2020, 9:39 am

Master list could look something like this :

Image

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#901

Post by Tim2460 » August 6th, 2020, 10:39 am


User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#902

Post by Melvelet » August 6th, 2020, 1:11 pm

So you'd just like to have country info for these or what else should be automatised?

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#903

Post by Tim2460 » August 6th, 2020, 1:17 pm

Easier would be to have everything/every collum from the jsons so i can cross reference with my excel !

User avatar
Fergenaprido
Donator
Posts: 4409
Joined: Jun 03, 2014
Location: Malaysia
Contact:

#904

Post by Fergenaprido » August 6th, 2020, 2:28 pm

I for one also keep track of the language field.

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#905

Post by Tim2460 » August 6th, 2020, 3:07 pm

Almost everything is interesting IMHO.

Maybe Writer is overkill, (especialy on Guardians of the galaxy exemple below) .

Not sure the 3 ratings will always be there, or in the same order ... that could be a probem

Rated and boxoffice are cool new stuff to keep track off

{
"Title":"Guardians of the Galaxy Vol. 2",
"Year":"2017",
"Rated":"PG-13",
"Released":"05 May 2017",
"Runtime":"136 min",
"Genre":"Action, Adventure, Comedy, Sci-Fi",
"Director":"James Gunn",

"Writer":"James Gunn, Dan Abnett (based on the Marvel comics by), Andy Lanning (based on the Marvel comics by), Steve Englehart (Star-Lord created by), Steve Gan (Star-Lord created by), Jim Starlin (Gamora and Drax created by), Stan Lee (Groot created by), Larry Lieber (Groot created by), Jack Kirby (Groot created by), Bill Mantlo (Rocket Raccoon created by), Keith Giffen (Rocket Raccoon created by), Steve Gerber (Howard the Duck created by), Val Mayerik (Howard the Duck created by)",
"Actors":"Chris Pratt, Zoe Saldana, Dave Bautista, Vin Diesel",
"Plot":"The Guardians struggle to keep together as a team while dealing with their personal family issues, notably Star-Lord's encounter with his father the ambitious celestial being Ego.",
"Language":"English",
"Country":"USA",
"Awards":"Nominated for 1 Oscar. Another 15 wins & 56 nominations.",

"Poster":"https://m.media-amazon.com/images/M/MV5 ... _SX300.jpg",
"Ratings":[{"Source":"Internet Movie Database","Value":"7.6/10"},{"Source":"Rotten Tomatoes","Value":"85%"},{"Source":"Metacritic","Value":"67/100"}],
"Metascore":"67",
"imdbRating":"7.6",
"imdbVotes":"545,424",
"imdbID":"tt3896198",
"Type":"movie",
"DVD":"22 Aug 2017",
"BoxOffice":"$389,804,217",
"Production":"Walt Disney Pictures",

"Website":"N/A",
"Response":"True"

}

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#906

Post by Tim2460 » August 7th, 2020, 12:06 pm

I'll be out of the city this weekend : hope we'll have some new scripts to play with next week if our team of scripters use the week end wisely ,)

User avatar
shugs
Donator
Posts: 587
Joined: Nov 15, 2014
Location: Bucharest, Romania
Contact:

#907

Post by shugs » August 8th, 2020, 7:49 am

I've imported the data from TMDb.

These are the movies that were not found there, a nice number of 150:
Missing tt codes from TMDbShow
tt0003037
tt0003165
tt0003952
tt0003930
tt4093826
tt0049669
tt1992236
tt0111053
tt0398010
tt0759927
tt0306457
tt0191897
tt0182309
tt2980346
tt0200426
tt2980304
tt1291144
tt0076136
tt2982244
tt2982248
tt2982274
tt2982290
tt2982222
tt2982258
tt2982226
tt2982198
tt2982206
tt2982250
tt2982204
tt2982186
tt2982182
tt2982208
tt0123996
tt0361434
tt2334570
tt2333736
tt0344877
tt2983828
tt2991548
tt0301283
tt0175031
tt0169609
tt0232655
tt5744734
tt2977438
tt2973624
tt2977472
tt2977410
tt2974334
tt2974370
tt1718200
tt1718202
tt3017376
tt1717631
tt2973638
tt2973558
tt1717667
tt3017384
tt1718169
tt2970656
tt2970634
tt0252322
tt0252982
tt0170146
tt0170610
tt0171773
tt2969474
tt0170312
tt2969230
tt2969290
tt2990498
tt2990502
tt2990508
tt2990510
tt2991088
tt2215028
tt2984002
tt1606327
tt0153616
tt0233210
tt2083140
tt2968820
tt2968506
tt2968382
tt2966768
tt2966632
tt0018121
tt0185449
tt2957914
tt0151874
tt0321468
tt2965948
tt2965952
tt0074931
tt0187169
tt0238070
tt2051808
tt2084769
tt2048800
tt2056721
tt2061820
tt2051846
tt2095617
tt2081249
tt2084875
tt2051944
tt3528366
tt2084916
tt2055776
tt0067709
tt2049531
tt0259033
tt0278360
tt0278797
tt0278361
tt0278148
tt3195904
tt3195978
tt2217857
tt0604374
tt0190029
tt0012702
tt0810983
tt6548410
tt6548178
tt0631310
tt2333356
tt0285294
tt1415212
tt2172496
tt8127644
tt3334996
tt7277634
tt11110378
tt11110316
tt1758429
tt4128194
tt0205837
tt0299352
tt0074756
tt3290674
tt3325544
tt3338106
tt8514090
tt2974896
tt11827072
tt8180398
tt10457346
tt10923778
tt0193602

Note that the genres are different from the ones on IMDb. They are:
GenresShow
Action
Action & Adventure
Adventure
Animation
Comedy
Crime
Documentary
Drama
Family
Fantasy
History
Horror
Kids
Music
Mystery
News
Reality
Romance
Sci-Fi & Fantasy
Science Fiction
Soap
Talk
Thriller
TV Movie
War
War & Politics
Western
Note that there's a type column with values feature or tv. TMDb has a separate id (sort of like a ttcode) for movies and tv stuff, so I added the column to avoid overlapping stuff.
Where the runtime or release year was missing, the value is 0 in the spreadsheet and the runtimes for tv entries are busted, they have a different way of storing the runtime for tv.

Anyway, here's the spreadsheet, let me know if you need anything else or if you see something that looks fishy
https://docs.google.com/spreadsheets/d/ ... sp=sharing

When I have some more free time I'll try to make a heroku app or something to keep this list automatically updated with changes, with different filters and what not.

User avatar
Fergenaprido
Donator
Posts: 4409
Joined: Jun 03, 2014
Location: Malaysia
Contact:

#908

Post by Fergenaprido » August 8th, 2020, 9:15 am

In these scripts that you folks are building, is there a way to highlight data changes? Some stuff we expect to change (rating, number of votes, etc.), but some stuff should be static (year, title, director). If there was a way to get notified or generate a report of changes, we could ensure that things stay correct and up-to-date on icm too (I'm thinking mainly of films with release dates that change back and forth from year to year, affecting eligibility in certain polls).

User avatar
shugs
Donator
Posts: 587
Joined: Nov 15, 2014
Location: Bucharest, Romania
Contact:

#909

Post by shugs » August 8th, 2020, 10:19 am

ICM uses IMDb as a datasource which may have a changes API, but you need to contact them https://developer.imdb.com/ . Speaking of which, I'm curious how ICM gets its data, since IMDb scraping is forbidden. :D
They also offer daily-refreshed datasets https://www.imdb.com/interfaces/ but the information there is incomplete (countries are missing, genres are limited to three entries), and from what I understood parsing them would be too much for the ICM servers.

I'd just change the datasource to TMDb (which Letterboxd uses). They might not have every obscure movie, but adding data is pretty easy and straight forward and unlike IMDb, it has a free and great API, which also includes a changes endpoint to automate updates: https://developers.themoviedb.org/3/cha ... hange-list

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#910

Post by Tim2460 » August 10th, 2020, 8:12 am

hello shugs : i really like the official name with the right alphabet ! Arabic looks amazing in Excel :=)

The country spelling and the genres are indeed différent from IMDB ... that could be (small) problem.

Anyway : good job

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#911

Post by Tim2460 » August 10th, 2020, 8:13 am

Melvelet : Do you want my Licence number for the site to grab more jsons ?

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#912

Post by Tim2460 » August 10th, 2020, 8:21 am

Fergenaprido : would you prefer all the countries in one collum only or separate country by X collums

Afghanistan,Ireland,Iran,Japan,Netherlands in 5 collums for example.

User avatar
Fergenaprido
Donator
Posts: 4409
Joined: Jun 03, 2014
Location: Malaysia
Contact:

#913

Post by Fergenaprido » August 10th, 2020, 9:04 am

Tim2460 wrote:
August 10th, 2020, 8:12 am
hello shugs : i really like the official name with the right alphabet ! Arabic looks amazing in Excel :=)
I think it would still be good to also have a Latin alphabet transcription of the original title, especially for those that don't have an English title.
Tim2460 wrote:
August 10th, 2020, 8:21 am
Fergenaprido : would you prefer all the countries in one collum only or separate country by X collums

Afghanistan,Ireland,Iran,Japan,Netherlands in 5 collums for example.
Whatever's easier - a single column would probably look better and give a more compact display.

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#914

Post by Melvelet » August 10th, 2020, 9:12 am

Does TMDb have have any limit on API requests and which one has the "better" data, OMDB or TMDB? The changes API from TMDB could come in handy
And in which way would our own database differ from TMDB/OMDB data if we also keep all the country information? Or rather: what's the value of the database vs. grabbing the (up-to-date) data of the source DB on demand?

EDIT: TMDB seems to have no request limit

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#915

Post by Tim2460 » August 10th, 2020, 9:28 am

there seems to be some différence on the country collum from OMDB/TMDB...

Very rare.

Also they don't seem to manage China/HonkGong/Taiwan the same way

OMDB seems to be closest to IMDB... at least on those 3

OMDB TMDB IMDB
Titanic 1997 USA, Mexico, Australia, Canada USA USA | Mexico | Australia | Canada
Gladiator 2000 USA, UK, Malta, Morocco UK,USA USA | UK | Malta | Morocco
Wo.hu.cang.long 2000 Taiwan, Hong Kong, USA, China China,Hong Kong,Taiwan,USA Taiwan | Hong Kong | USA | China
Last edited by Tim2460 on August 10th, 2020, 9:35 am, edited 1 time in total.

User avatar
shugs
Donator
Posts: 587
Joined: Nov 15, 2014
Location: Bucharest, Romania
Contact:

#916

Post by shugs » August 10th, 2020, 9:32 am

Melvelet wrote:
August 10th, 2020, 9:12 am
Does TMDb have have any limit on API requests and which one has the "better" data, OMDB or TMDB? The changes API from TMDB could come in handy
And in which way would our own database differ from TMDB/OMDB data if we also keep all the country information? Or rather: what's the value of the database vs. grabbing the (up-to-date) data of the source DB on demand?

EDIT: TMDB seems to have no request limit
They used to have a request limit until a few months ago, but they removed it. OMDB has the better data since it takes it directly from IMDb, but, I don't know, it doesn't seem to be actively maintained and asking for money for non-proprietary data is fishy at best. I kinda expect IMDb to take action against it. :ermm: It also doesn't have a way to monitor changes, but a full refresh of all official movies once in a while shouldn't be that taxing anyway.

User avatar
shugs
Donator
Posts: 587
Joined: Nov 15, 2014
Location: Bucharest, Romania
Contact:

#917

Post by shugs » August 10th, 2020, 9:38 am

Tim2460 wrote:
August 10th, 2020, 9:28 am
there seems to be some différence on the country collum from OMDB/TMDB...

Very rare.

Also they don't seem to manage China/HonkGong/Taiwan the same way

OMDB seems to be closest to IMDB... at least on those 3

OMDB TMDB IMDB
Titanic 1997 USA, Mexico, Australia, Canada USA USA | Mexico | Australia | Canada
Gladiator 2000 USA, UK, Malta, Morocco UK,USA USA | UK | Malta | Morocco
Wo.hu.cang.long 2000 Taiwan, Hong Kong, USA, China China,Hong Kong,Taiwan,USA Taiwan | Hong Kong | USA | China
Ah, yes, Titanic, the Mexican movie. :D

I think TMDb has a different philosophy when it comes to countries. Looked briefly through their forums but couldn't find anything on how they decide what countries to use. But, based on your examples, I like their data more. :D

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#918

Post by Tim2460 » August 10th, 2020, 9:45 am

I think both site will bring interesting info.

Our main use is to recreate an master list with up to date info from IMDB, ICM and maybe now OMDB & TMDB.
This list will be used on the result threads of our pools (maybe the Movie poster together an screenshoot will be cool for example).

Problem is to actually "fill" the full extract from OMDB and TMDB and choose the right DB format for both ... i guess CSV would be the easier.
Can be done in one shot for TMDB with shugs method.
Got to be done by packs of 1000 with melvelet script.

Maybe an script that grabbs only films that are not on the existing csv will do the trick.

If it"s too compliquated, i'll take a look to code it myself on Ruby.

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#919

Post by Melvelet » August 10th, 2020, 10:01 am

I'd argue that it's even better to have the filming location-style countries removed for our purposes, so I'd have a strong preference towards TMDB.
Anyway, I still wonder if it's the best approach to maintain a permanently out-of-date database dump in a spreadsheet format vs. creating an API based tool for the specific tasks (plus potentially a screenshot database) :D

A month-old TMDB database dump exists here for example: https://www.kaggle.com/edgartanaka1/tmd ... and-series
Last edited by Melvelet on August 10th, 2020, 10:11 am, edited 1 time in total.

User avatar
Fergenaprido
Donator
Posts: 4409
Joined: Jun 03, 2014
Location: Malaysia
Contact:

#920

Post by Fergenaprido » August 10th, 2020, 10:03 am

Letterboxd also uses tmdb for their data, and their country data is filled with gaping holes (among other issues). I'd rather have to filter out extra countries than have to search to fill in missing countries. With all the problems that imdb has, I still trust their data more (and for countries you can also place them in order... for tmdb it looks like it just lists them alphabetically?).

Post Reply