Welcome to the ICM Forum. If you have an account but have trouble logging in, or have other questions, see THIS THREAD.
Podcast: Talking Images (Episode 16 released September 13th)
Polls: Romance (Results), 1951 (Results), 500<400 (Sep 23rd), 2008 (Oct 4th)
Challenges: Animation, Silent Era, Russia/USSR
Film of the Week: Durak, October nominations (Sep 25th)
World Cup S4: QF Schedule, Match QFB: India vs Greece (Sep 20th), Match QFC: Germany vs Italy (Oct 1st)

User-made scripts and apps etc

Post Reply
User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

Re: User-made scripts and apps etc

#921

Post by Tim2460 » August 10th, 2020, 12:19 pm

As i said before, my view is ... get every info you can ;)

Box office and poster link info is cool from OMDB, original tittle is cool from TMDB etc etc ...

Ideal solution would be for the 2 scripts to use the same imput with the ttnumbers, with both checking the ttnumbers against their own CSV database to check if an API-call is nedeed.

Each month, i'll then import the new official films to the master Excel sheet and use the scripts and my ICM/IMDB access to fill the collums.

We would also need the final pool result (each month?) to fill the gap there as well.

If we want to recheck everything or part of the extracts ... just delete the corresponding lines from the csv database.

I can manage that workflow on my monthly rebuilding of the official lists.

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#922

Post by Melvelet » August 11th, 2020, 11:00 am

The script could easily combine multiple/all sources. I don't know how the ICM infos (# of lists, checks) is gathered but if you have a file with that info, it can simply be read and processed.
We'd just need to define which infos should have which (main) source and which info is not auto-updated (Country? Aka only initial import and then manually deleting extra countries?)

And a genuine question: shall certain parts not be automated because a certain degree of manual work (that could be automated) is part of the fun that I would take away with too much automation? And is the spreadsheet's purpose to serve as data source for other tasks (poll results, official movies per country) or does it have value/actual users on its own?

Because it's not really more effort to have a tool where you just have a list of IMDb links (or poll results) as input that will give you a csv file and/or correctly formatted poll results as output. With a screenshot (and country data) spreadsheet as only manual work instead of regular manual work to maintain a database copy (that might even result in outdated data for the poll results). Although even the manual spreadsheet maintenance work could be reduced to next-to-nothing (except manually adding screencaps and countries)

PS: Does OMDB actually not supply the original title? Oo
PPS: Monthly updating of OMDB data in the spreadsheet would of course have the 1000 API requests problem

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#923

Post by Tim2460 » August 11th, 2020, 1:42 pm

Let's see...

Every month, for my personal excel spreadsheet maintenance and creating the official lists @ iCM i use mfj314 java util that download every official lists.

Theses 206 files are then processed by an Ruby script that gives me this file/csv.

https://we.tl/t-6dFSsFJZxw

it shows all the films in lines and if that movie is on each list as collum.

I then process that csv on my excel to add the new official films every month.

Old official films stays on my main excel : there is an huge chance they'll be back eventually next month.

That's my use.

Pool result posting is a little different.
We already have an "master" excel file with the ttnumber and URl to the screenshot.
And complcated excels that creates the actual posts to be umploaded t othe forum.
Theses need to be filled with all kind of info, mainly from IMDB & ICM lists extracted.

Creating IMDB lists, especialy with huge number of film is quite time consuming.
ICM Lists are a little faster to create... if every film is already on ICM ;).

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#924

Post by Tim2460 » August 11th, 2020, 1:57 pm

Here's what would be interesting to have extracted on both site IMHO :

Extract ? OMDB
X Title Guardians of the Galaxy Vol. 2,
X Year 2017
X Rated PG-13,
X Released 05 May 2017,
X Runtime 136 min,
X Genre Action, Adventure, Comedy, Sci-Fi,
X Director James Gunn,
Writer Too much
X Actors Chris Pratt, Zoe Saldana, Dave Bautista, Vin Diesel,
X Plot The Guardians struggle to keep together as a team while dealing with their personal family issues, notably Star-Lord's encounter with his father the ambitious celestial being Ego.,
X Language English,
X Country USA,
X Awards Nominated for 1 Oscar. Another 15 wins & 56 nominations.,
X Poster "https //m.media-amazon.com/images/M/MV5 ... _SX300.jpg,"
Ratings hard to Parse
X Metascore 67
X imdbRating 7.6,
X imdbVotes 545,424,
X imdbID tt3896198,
X Type movie,
X DVD 22 Aug 2017,
X BoxOffice $389,804,217,
X Production Walt Disney Pictures,
Website N/A,
Response True
TMDB
X imdb_id tt3896198
tmdb_id 283995
type feature
X original_title Guardians of the Galaxy Vol. 2
X english_title Guardians of the Galaxy Vol. 2
X year 2017
runtime 137
directors James Gunn
genres Action,Adventure,Comedy,Science Fiction
countries United States of America

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#925

Post by Tim2460 » August 11th, 2020, 2:03 pm

The Pool result...

Code: Select all

=CONCATENER("[size=125][size=125][size=125][size=125][b]#";W108;"[/b][/size](";;SI(V108="NEW";V108;SI(V108=0;CONCATENER(" – , #";X108);SI(V108<0;CONCATENER("[b][color=#ee4a2d]⇩";ABS(V108);"[/color][/b], #";X108);CONCATENER("[b][color=#090]⇧";V108;"[/color][/b], #";X108))));;") [size=125][url=";I108;"]";C108;" (";G108;")[/url][/size][/size]";SI(U108="";U108;CONCATENER(CAR(10);"[";U108;"]"));"[/size][/size]";CAR(10);AE108;CAR(10);"[size=125][size=125]Directed by: [size=125]";E108;"[/size][/size][/size]";CAR(10);"[size=125][size=125]([b]";ARRONDI(J108;2);"[/b] Pts, [b]";K108;"[/b] Votes)[/size] , Top 1–10: ";M108;"–";N108;"[/size]";CAR(10);"[hr] [/hr][size=125][url=";Q108;"]ICheckMovies[/url]: [b]";R108;"[/b] Checks , [b]";S108;"[/b] Favourites , [b]";T108;"[/b] Official lists[/size][spoiler=List of Voters:]";AG108;"[/Spoiler]")
makes that nice post :

#107(⇩41, #66) I Walked with a Zombie (1943)
Image
Directed by: Jacques Tourneur
(704.94 Pts, 15 Votes) , Top 1–10: –1
ICheckMovies: 2799 Checks , 194 Favourites , 8 Official lists
List of Voters:Show
bal3x (#74)
burneyfan(unranked)
Gordon_Gekko (#74)
Gorro (#155)
Ivan0716 (#113)
jeff_v (#117)
mathiasa (#36)
Obgeoff (#50)
Opio (#76)
outdoorcats (#103)
peeptoad (#11)
perceval (#8)
psychotronicbeatnik(unranked)
rnilsson19 (#14)
XxXApathy420XxX (#48)

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#926

Post by Tim2460 » August 11th, 2020, 2:04 pm

Having even more info would be very cool.

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#927

Post by Tim2460 » August 14th, 2020, 1:39 pm

For every pool manager

PA, Joachimt, Lonewolf etc etc

... can you take a look at this excel and let me know what you think ??

https://we.tl/t-hvpKObdRuc

On the extract Tab, you'll see what info we would need to extract, and from what Site so our scripter can know what to include on the extracts from OMDB & TMDB

The Most can be find on the OMDB site ...
If i got it right we would only need the ICM extract for ICMCheckedcount, officialtoplistcount and akatitle.
No need to make an IMDB List anymore... witch is cool as it's the longuest operation.

User avatar
joachimt
Donator
Posts: 32235
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#928

Post by joachimt » August 14th, 2020, 2:05 pm

I'm not sure what question you want me to answer, because I didn't really follow the discussion above. Also, it's been a while since I did a poll, so maybe Lonewolf should answer your questions.
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#929

Post by Tim2460 » August 17th, 2020, 12:27 pm

@ Melvelet : can you PM me the script that would get imdb ttnumer as input ant output everything has csv ?

I'd like to fill the excel file with extra data... even if it's 1000 lines at a time.

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#930

Post by Melvelet » August 18th, 2020, 11:18 am

Sorry, I was busy with studying and the <400 challenge duties. I'll be working on the this tool a bit this week. I'll write a longer post in the evening :)

User avatar
Lonewolf2003
Donator
Posts: 9633
Joined: Dec 29, 2012
Contact:

#931

Post by Lonewolf2003 » August 18th, 2020, 4:28 pm

Tim2460 wrote:
August 14th, 2020, 1:39 pm
For every pool manager

PA, Joachimt, Lonewolf etc etc

... can you take a look at this excel and let me know what you think ??

https://we.tl/t-hvpKObdRuc

On the extract Tab, you'll see what info we would need to extract, and from what Site so our scripter can know what to include on the extracts from OMDB & TMDB

The Most can be find on the OMDB site ...
If i got it right we would only need the ICM extract for ICMCheckedcount, officialtoplistcount and akatitle.
No need to make an IMDB List anymore... witch is cool as it's the longuest operation.
I think you have all the information you would want for a poll.

For polls the following information is must-have to post in the result posts:
Original title (in western alphabet (or whatever its called)
Aka title
Director
Year
iCM Check count
iCM official list count
iCM link
IMDb link
rankdifference (or previous rank) is must-have for recurring polls.

The following is must-have to, but mostly for stat afterwards or depending on the poll subject in the results post.
Genres
Countries
Type (short, miniseries etc)
Language (could be usefull/interesting in some country/region polls)

Nice to have, but not necessary:
TMDB Original title


Would it be possible to remove the dots in the TMDB titles? So those don't show up in the results posts.

Did you get the screenshot urls from the list I gave to Ferge? If not I can message you my list and you could combine it. Maybe joachimt can mail his too.
Last edited by Lonewolf2003 on August 18th, 2020, 4:43 pm, edited 2 times in total.

User avatar
Lonewolf2003
Donator
Posts: 9633
Joined: Dec 29, 2012
Contact:

#932

Post by Lonewolf2003 » August 18th, 2020, 4:29 pm

Tim2460 wrote:
August 11th, 2020, 1:42 pm
Let's see...

Every month, for my personal excel spreadsheet maintenance and creating the official lists @ iCM i use mfj314 java util that download every official lists.

Theses 206 files are then processed by an Ruby script that gives me this file/csv.

https://we.tl/t-6dFSsFJZxw

it shows all the films in lines and if that movie is on each list as collum.

I then process that csv on my excel to add the new official films every month.

Old official films stays on my main excel : there is an huge chance they'll be back eventually next month.

That's my use.

Pool result posting is a little different.
We already have an "master" excel file with the ttnumber and URl to the screenshot.
And complcated excels that creates the actual posts to be umploaded t othe forum.
Theses need to be filled with all kind of info, mainly from IMDB & ICM lists extracted.

Creating IMDB lists, especialy with huge number of film is quite time consuming.
ICM Lists are a little faster to create... if every film is already on ICM ;).
Which unfortunately rarely is the case ;)

User avatar
Lonewolf2003
Donator
Posts: 9633
Joined: Dec 29, 2012
Contact:

#933

Post by Lonewolf2003 » August 18th, 2020, 4:34 pm

Tim2460 wrote:
August 11th, 2020, 2:04 pm
Having even more info would be very cool.
For readability and clarity I would recommend not to overflow a post with too much information. More is not always better ;) Of course in some cases some more info could be nice, like countries in region polls.

User avatar
Lonewolf2003
Donator
Posts: 9633
Joined: Dec 29, 2012
Contact:

#934

Post by Lonewolf2003 » August 18th, 2020, 4:40 pm

Unrelated to polls posts: if the sheets is going to have run-times, would it be possible to cross-check the list with every movie without a runtime on iCM. Updating the run-time info on iCM has to be done manually (by clicking the update button in the beta). But the problem is that most movies with a runtime also don't have a runtime on IMDb. Since iCM gets its information from that, updating the info on iCM for movies with no runtime on IMDb is useless. So it would be great if there would be a comparison between the movies on iCM with no runtime and those on IMDb with runtime, so we can see which movies actually are useful to update.

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#935

Post by Tim2460 » August 18th, 2020, 8:30 pm

Lone : that would be very easy to cross ref the 2 runtime : will do tomorrow!

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#936

Post by Melvelet » August 18th, 2020, 10:24 pm

Tim2460 wrote:
August 18th, 2020, 8:30 pm
Lone : that would be very easy to cross ref the 2 runtime : will do tomorrow!
Oh, do we have a way to get a list of the movies missing the runtime info? I wrote a post related to that but then I hung out with my flatmates and got drunk instead :whistling:

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#937

Post by Tim2460 » August 19th, 2020, 8:28 am

Tim2460 wrote:
August 18th, 2020, 8:30 pm
Lone : that would be very easy to cross ref the 2 runtime : will do tomorrow!
:facepalm: I really thaught it would be easy ... but the ICM Extract don't have the runtime :down:

Only collums we can cross ref between ICM and IMDB extracts are tittle and Year of release.

Year was interesting for Fergenaprido !

If you wana play with the Datamining excel file included... https://we.tl/t-LG2OeqaBf5
I count 237 Films that have a bad year
and 232 with an different tittle

--> thats the D & G collum of the ICm TAB filtered as "Update" for tittle & Year respectively.

You can also check films that don't have Runtime on the IMDB TAB.

The ICM and IMDB extract are from 2020/05/20. And it contains the Official Films and other i have interestet collecting (Peary's Book essantialy)

Maybe Ebby keeps an excel list of the ICM film he found without Runtime ?

User avatar
Fergenaprido
Donator
Posts: 4409
Joined: Jun 03, 2014
Location: Malaysia
Contact:

#938

Post by Fergenaprido » August 19th, 2020, 9:38 am

Thanks Tim. I'll check out those films with a year or title mismatch.

User avatar
Lonewolf2003
Donator
Posts: 9633
Joined: Dec 29, 2012
Contact:

#939

Post by Lonewolf2003 » August 19th, 2020, 10:06 am

Tim2460 wrote:
August 19th, 2020, 8:28 am
Tim2460 wrote:
August 18th, 2020, 8:30 pm
Lone : that would be very easy to cross ref the 2 runtime : will do tomorrow!
:facepalm: I really thaught it would be easy ... but the ICM Extract don't have the runtime :down:

Only collums we can cross ref between ICM and IMDB extracts are tittle and Year of release.

Year was interesting for Fergenaprido !

If you wana play with the Datamining excel file included... https://we.tl/t-LG2OeqaBf5
I count 237 Films that have a bad year
and 232 with an different tittle

--> thats the D & G collum of the ICm TAB filtered as "Update" for tittle & Year respectively.

You can also check films that don't have Runtime on the IMDB TAB.

The ICM and IMDB extract are from 2020/05/20. And it contains the Official Films and other i have interestet collecting (Peary's Book essantialy)

Maybe Ebby keeps an excel list of the ICM film he found without Runtime ?
Thanks for trying! I will try if we have a list somewhere with all the movies with runtime.

User avatar
Melvelet
Posts: 2161
Joined: Mar 29, 2013
Contact:

#940

Post by Melvelet » August 20th, 2020, 3:21 pm

Sent you a PM, Tim, in case you haven't seen it

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#941

Post by Tim2460 » August 20th, 2020, 3:46 pm

Yes saw it!

Will be difficult to adapt it o grabbing everything but I'll take a look

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#942

Post by Tim2460 » August 21st, 2020, 9:35 am

Take a look at that ....

https://www.wikidata.org/wiki/Q2345

Open Source by design...

It's the database behind OMDB.ORG and OMDBAPI.org

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#943

Post by Tim2460 » August 21st, 2020, 9:52 am

Some of films got that tag ....

Image

Interesting isn't it ?

User avatar
Tim2460
Posts: 2519
Joined: Oct 01, 2018
Location: Dijon, France
Contact:

#944

Post by Tim2460 » September 11th, 2020, 12:27 pm

Ok

Managed to run my first Python scripts thx to Melvelet Help.

About time at 49 years old ;)

I'm grabbing the info from OMDB :

imdb https://www.imdb.com/title/tt0052357/
tt0052357
icm https://www.icheckmovies.com/search/mov ... =tt0052357
Title Vertigo
Year 1958
AKA
TMDB Original
TMDB english
Type movie
IMDB 8.3
Run 128
Type Mystery, Romance, Thriller
Release.Date 1958-05-09
ICM C 43488
URL IMDB
URL PTP
URL ICM
Got X
Add
Nb 39
Director Alfred.Hitchcock
Number 00001
Name 00001.Vertigo.1958.tt0052357.mkv
Size 16.0 GiB
17 165 130 537
Lenght 2 h 8 min
Larg 1920
Ht 1080
imdbID tt0052357
Title Vertigo
Year 1958
Rated PG
Released 22 May 1958
Runtime 128 min
Genre Mystery, Romance, Thriller
Director Alfred Hitchcock
Writer Alec Coppel (screenplay by), Samuel A. Taylor (screenplay by), Pierre Boileau (based on the novel "D'Entre Les Morts" by), Thomas Narcejac (based on the novel "D'Entre Les Morts" by)
Actors James Stewart, Kim Novak, Barbara Bel Geddes, Tom Helmore
Plot A former police detective juggles wrestling with his personal demons and becoming obsessed with a hauntingly beautiful woman.
Language English
Country USA
Awards Nominated for 2 Oscars. Another 8 wins & 5 nominations.
Poster https://m.media-amazon.com/images/M/MV5 ... _SX300.jpg
Metascore 100
imdbRating 8.3
imdbVotes 351,317
imdbID tt0052357
Type movie
BoxOffice N/A
Production Paramount Pictures
Website N/A

Post Reply