Welcome to the ICM Forum. If you have an account but have trouble logging in, or have other questions, see THIS THREAD.
ICMForum's Favourite Movies 2019 Edition (Nominations: Midnight Saturday Mar 23rd)
Polls: Academy Awards (Mar 22nd), 1921 awards (Apr 3rd), Children (Apr 7th), 1932 (Apr 14th), DtC Voting (Apr 25th)
Challenges: French, War, Female Directors
World Cup: Deadline for monty and filmbantha (Mar 30th)
Film of the Week: Una pura formalità

User-made scripts and apps etc

Post Reply
User avatar
PeacefulAnarchy
Moderator
Posts: 21707
Joined: May 08, 2011
Contact:

User-made scripts and apps etc

#561

Post by PeacefulAnarchy » December 13th, 2017, 1:42 am

I'm waiting a bit to update since I expect imdb to keep fiddling with things, but I'm glad themagician is on top of this, that script is invaluable.

mjf314
Moderator
Posts: 10596
Joined: May 08, 2011
Contact:

#562

Post by mjf314 » December 13th, 2017, 11:48 pm

Here's an updated version of filmcounter8. I think it should work with the new IMDb list format, but I didn't have time to test it, so let me know if it works. The only change I made was changing const to Const.

If it doesn't work then I'll look at it more closely on Thursday or Friday.

Attachments:
Last edited by mjf314 on December 15th, 2017, 9:03 am, edited 1 time in total.

User avatar
joachimt
Donator
Posts: 27830
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#563

Post by joachimt » December 14th, 2017, 7:41 am

Thanks. I'll test it this evening on the LGBT-results.
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
joachimt
Donator
Posts: 27830
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#564

Post by joachimt » December 14th, 2017, 9:55 pm

Listdownloader isn't working anymore. I tried redownloading the lists from the road trip poll, because I know it worked there a few weeks ago, but now it only downloads iCM-lists. So the changes on IMDb also effect that program.

I'll manually download the LGBT-lists, so I can continue for now.
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
joachimt
Donator
Posts: 27830
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#565

Post by joachimt » December 14th, 2017, 10:33 pm

Or maybe my IMDb-cookie changed?
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
joachimt
Donator
Posts: 27830
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#566

Post by joachimt » December 14th, 2017, 10:56 pm

mjf314 on Dec 13 2017, 04:48:42 PM wrote:Here's an updated version of filmcounter8. I think it should work with the new IMDb list format, but I didn't have time to test it, so let me know if it works. The only change I made was changing const to Const.

If it doesn't work then I'll look at it more closely on Thursday or Friday.
It doesn't work. It only takes iCM-lists into account, so the filmcount.csv includes only the points by mathiasa.

Here is everything that I have in the LGBT-folder, so you can try for yourself and hopefully fix it:
https://we.tl/Ct0F3vieoz
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

mjf314
Moderator
Posts: 10596
Joined: May 08, 2011
Contact:

#567

Post by mjf314 » December 15th, 2017, 8:54 am

It looks like the IMDb list export URL has been changed, so I'll have to update the Listdownloader program.

In the IMDb lists, the IMDb URLs are no longer surrounded by quotes, so I think that's why it doesn't work.

Edit: I replaced filmcounter8.1 in the post above with a fixed version. It seems to be working now.
Last edited by mjf314 on December 15th, 2017, 9:04 am, edited 1 time in total.

User avatar
joachimt
Donator
Posts: 27830
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#568

Post by joachimt » December 15th, 2017, 12:38 pm

IMDb List Helper is not working again. Both with Tampermonkey and ViolentMonkey I see no field to enter URLs.
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
bal3x
Donator
Posts: 12772
Joined: May 26, 2011
Contact:

#569

Post by bal3x » December 15th, 2017, 3:11 pm

Seems IMDb keep changing stuff... what a mess.

themagician
Posts: 3889
Joined: Jun 18, 2011
Location: Finland
Contact:

#570

Post by themagician » December 15th, 2017, 10:20 pm

joachimt on Dec 15 2017, 05:38:52 AM wrote:IMDb List Helper is not working again. Both with Tampermonkey and ViolentMonkey I see no field to enter URLs.
Did it stop working when you created a new list? I just released 2.4.1 that fixes that.

User avatar
joachimt
Donator
Posts: 27830
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#571

Post by joachimt » December 15th, 2017, 10:54 pm

themagician on Dec 15 2017, 03:20:23 PM wrote:
joachimt on Dec 15 2017, 05:38:52 AM wrote:IMDb List Helper is not working again. Both with Tampermonkey and ViolentMonkey I see no field to enter URLs.
Did it stop working when you created a new list? I just released 2.4.1 that fixes that.
Yes, I tried creating a new list. I'll try the upgrade tomorrow. Thanks!
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
monk-time
Posts: 1334
Joined: Mar 23, 2015
Contact:

#572

Post by monk-time » December 16th, 2017, 10:14 pm

mightysparks on Sep 26 2016, 10:53:12 PM wrote:I keep having issues with MyMovies Enhancer... is anyone here able to like rewrite it or something so that it works properly?
I think I've been having the same issue with it for the last year or so: nine times out of ten the progress bar freezes because some lists don't download correctly - for some bizarre reason on occasion IMDb returns HTML instead of CSV when the script tries to export a list. I can't imagine using IMDb without it, so I have rewrote it completely last month but didn't test it well enough, and that was before they went on an updating spree. I think I'll be able to find time to finish and publish it sometime this or next month.
ICM profile | My userscripts for IMDb/iCM | iCheckMovies Enhanced by themagician: call it direct, call it collect, but CALL IT TODAY!

themagician
Posts: 3889
Joined: Jun 18, 2011
Location: Finland
Contact:

#573

Post by themagician » December 17th, 2017, 3:44 pm

I managed to get MyMovies Enhancer working. Let me know if there's any bugs.

edit: Watchlist not supported for now. Maybe later if anyone needs it.

edit: Wait for official/monk-time update
Last edited by themagician on December 17th, 2017, 8:09 pm, edited 1 time in total.

User avatar
flaiky
Posts: 1194
Joined: Feb 04, 2017
Location: London UK
Contact:

#574

Post by flaiky » December 17th, 2017, 4:18 pm

Can any of the tech wizzes here make it possible for the poll-counter-script-thingy to work with Letterboxd lists?
Let the ashes fly
ICM | Letterboxd | All-time stats

User avatar
monk-time
Posts: 1334
Joined: Mar 23, 2015
Contact:

#575

Post by monk-time » December 17th, 2017, 5:38 pm

themagician on Dec 17 2017, 08:44:01 AM wrote:I managed to get MyMovies Enhancer working. Let me know if there's any bugs.
I see you have used v1.35, Ricardo maintains his script on openuserjs, last version is 1.39. That version didn't have any issues with .csv parsing (well, until the recent changes): you can just split a line by '","', if any field contains a double quote character it would be escaped and not match this substring. And even then I haven't managed to find a single .csv from IMDb that actually has " in any field.

The problem was not with that, it's with the script not retrying on erroneous responses that IMDb often gave when trying to export multiple lists in batch during the last year or two. But it might as well have been fixed with IMDb's recent list exporting overhaul.

Anyway, I have added retrying for my fork, I just need some time to test it in all engines/browsers and publish it.
ICM profile | My userscripts for IMDb/iCM | iCheckMovies Enhanced by themagician: call it direct, call it collect, but CALL IT TODAY!

themagician
Posts: 3889
Joined: Jun 18, 2011
Location: Finland
Contact:

#576

Post by themagician » December 17th, 2017, 6:11 pm

monk-time on Dec 17 2017, 10:38:23 AM wrote:
themagician on Dec 17 2017, 08:44:01 AM wrote:I managed to get MyMovies Enhancer working. Let me know if there's any bugs.
I see you have used v1.35, Ricardo maintains his script on openuserjs, last version is 1.39. That version didn't have any issues with .csv parsing (well, until the recent changes): you can just split a line by '","', if any field contains a double quote character it would be escaped and not match this substring. And even then I haven't managed to find a single .csv from IMDb that actually has " in any field.

The problem was not with that, it's with the script not retrying on erroneous responses that IMDb often gave when trying to export multiple lists in batch during the last year or two. But it might as well have been fixed with IMDb's recent list exporting overhaul.

Anyway, I have added retrying for my fork, I just need some time to test it in all engines/browsers and publish it.
The new CSV format only uses " if it has commas, e.g.:

Code: Select all

4,tt0017048,2017-12-15,2017-12-15,,Kurutta ippêji,http://www.imdb.com/title/tt0017048/,movie,7.6,70,1926,"Drama, Horror",1807,1926-09-24,Teinosuke Kinugasa
So you can't split by '","'. If you try to split by ',' then that fails when the title has a comma.

Also the export URLs are different. They're also different for ratings and lists so that needs a workaround as well.

themagician
Posts: 3889
Joined: Jun 18, 2011
Location: Finland
Contact:

#577

Post by themagician » December 17th, 2017, 6:58 pm

themagician on Dec 17 2017, 08:44:01 AM wrote:I managed to get MyMovies Enhancer working. Let me know if there's any bugs.

edit: Watchlist not supported for now. Maybe later if anyone needs it.
I added some changes from v1.39, updating recommended.

User avatar
monk-time
Posts: 1334
Joined: Mar 23, 2015
Contact:

#578

Post by monk-time » December 17th, 2017, 7:07 pm

themagician on Dec 17 2017, 11:11:33 AM wrote:So you can't split by '","'. If you try to split by ',' then that fails when the title has a comma.
Damn, haven't looked into their new format yet.

I was looking into their mess with export URLs yesterday, the URL to export ratings is always the same, but it looks like you can't get around making an additional request to imdb.com/user/urXXXXXXXX/watchlist to extract the export URL from there, they now store the watchlist as a ls00XXXXXXX list as well.
Last edited by monk-time on December 17th, 2017, 7:09 pm, edited 1 time in total.
ICM profile | My userscripts for IMDb/iCM | iCheckMovies Enhanced by themagician: call it direct, call it collect, but CALL IT TODAY!

mjf314
Moderator
Posts: 10596
Joined: May 08, 2011
Contact:

#579

Post by mjf314 » December 17th, 2017, 11:03 pm

flaiky on Dec 17 2017, 09:18:06 AM wrote:Can any of the tech wizzes here make it possible for the poll-counter-script-thingy to work with Letterboxd lists?
Is it possible to export Letterboxd lists? Do they include IMDb URLs?

User avatar
maxwelldeux
Donator
Posts: 4893
Joined: Jun 07, 2016
Location: Seattle-ish, WA, USA
Contact:

#580

Post by maxwelldeux » December 18th, 2017, 12:08 am

mjf314 on Dec 17 2017, 04:03:04 PM wrote:
flaiky on Dec 17 2017, 09:18:06 AM wrote:Can any of the tech wizzes here make it possible for the poll-counter-script-thingy to work with Letterboxd lists?
Is it possible to export Letterboxd lists? Do they include IMDb URLs?
Related (selfish) question for trakt.tv... the answers are yes and yes.

Can you share what columns/values are used in the program so that I could massage my list format into something that can be read by the program?

User avatar
flaiky
Posts: 1194
Joined: Feb 04, 2017
Location: London UK
Contact:

#581

Post by flaiky » December 18th, 2017, 12:17 am

mjf314 on Dec 17 2017, 04:03:04 PM wrote:
flaiky on Dec 17 2017, 09:18:06 AM wrote:Can any of the tech wizzes here make it possible for the poll-counter-script-thingy to work with Letterboxd lists?
Is it possible to export Letterboxd lists? Do they include IMDb URLs?
Hmm, looks like only the creator can export them and even then, they include Ltbxd links instead of imdb links.
Let the ashes fly
ICM | Letterboxd | All-time stats

mjf314
Moderator
Posts: 10596
Joined: May 08, 2011
Contact:

#582

Post by mjf314 » December 18th, 2017, 1:30 am

maxwelldeux on Dec 17 2017, 05:08:02 PM wrote:
mjf314 on Dec 17 2017, 04:03:04 PM wrote:
flaiky on Dec 17 2017, 09:18:06 AM wrote:Can any of the tech wizzes here make it possible for the poll-counter-script-thingy to work with Letterboxd lists?
Is it possible to export Letterboxd lists? Do they include IMDb URLs?
Related (selfish) question for trakt.tv... the answers are yes and yes.

Can you share what columns/values are used in the program so that I could massage my list format into something that can be read by the program?
My program doesn't look at columns. It just searches for IMDb URLs.

If it's an iCM list it searches for this regex: "http://www.imdb.com/title/tt\\d{7}/"

If it's an IMDb list it searches for this regex: "(?<=,)http://www.imdb.com/title/tt&#092;&#092;d{7}/"

It's an IMDb list if the first line of the file contains "Const".

The (?<=,) means that it must be preceded by a comma (the purpose is to make it ignore IMDb URLs in comments, which are unlikely to be preceded by commas).

User avatar
maxwelldeux
Donator
Posts: 4893
Joined: Jun 07, 2016
Location: Seattle-ish, WA, USA
Contact:

#583

Post by maxwelldeux » December 18th, 2017, 1:50 am

mjf314 on Dec 17 2017, 06:30:16 PM wrote:
maxwelldeux on Dec 17 2017, 05:08:02 PM wrote:
mjf314 on Dec 17 2017, 04:03:04 PM wrote:Is it possible to export Letterboxd lists? Do they include IMDb URLs?
Related (selfish) question for trakt.tv... the answers are yes and yes.

Can you share what columns/values are used in the program so that I could massage my list format into something that can be read by the program?
My program doesn't look at columns. It just searches for IMDb URLs.

If it's an iCM list it searches for this regex: "http://www.imdb.com/title/tt&#092;&#092;d{7}/"

If it's an IMDb list it searches for this regex: "(?<=,)http://www.imdb.com/title/tt&#092;&#092;d{7}/"

It's an IMDb list if the first line of the file contains "Const".

The (?<=,) means that it must be preceded by a comma (the purpose is to make it ignore IMDb URLs in comments, which are unlikely to be preceded by commas).
So if I uploaded a list (csv) that had a single column of IMDB urls in there, your program would be able to read it?

User avatar
monk-time
Posts: 1334
Joined: Mar 23, 2015
Contact:

#584

Post by monk-time » December 18th, 2017, 2:27 am

flaiky on Dec 17 2017, 05:17:24 PM wrote:Hmm, looks like only the creator can export them and even then, they include Ltbxd links instead of imdb links.
This formula for Google Sheets should extract an IMDb link from Letterboxd:

Code: Select all

=IMPORTXML(A1, "//span[@class='film-ext-links']/a[1]/@href")
And for trakt.tv, the site doesn't seem to have a fixed position for IMDb links, so this will dump all external links (the previous one can be modified similarly to also fetch TMDB links):

Code: Select all

=TRANSPOSE(IMPORTXML(A1, "//ul[@class='external']/li/a/@href"))
In case they break it's easy to fix them using the Inspector tool in the browser (press F12) and any XPath manual.
ICM profile | My userscripts for IMDb/iCM | iCheckMovies Enhanced by themagician: call it direct, call it collect, but CALL IT TODAY!

mjf314
Moderator
Posts: 10596
Joined: May 08, 2011
Contact:

#585

Post by mjf314 » December 18th, 2017, 2:36 am

maxwelldeux on Dec 17 2017, 06:50:59 PM wrote:
mjf314 on Dec 17 2017, 06:30:16 PM wrote:
maxwelldeux on Dec 17 2017, 05:08:02 PM wrote:Related (selfish) question for trakt.tv... the answers are yes and yes.

Can you share what columns/values are used in the program so that I could massage my list format into something that can be read by the program?
My program doesn't look at columns. It just searches for IMDb URLs.

If it's an iCM list it searches for this regex: "http://www.imdb.com/title/tt&#092;&#092;d{7}/"

If it's an IMDb list it searches for this regex: "(?<=,)http://www.imdb.com/title/tt&#092;&#092;d{7}/"

It's an IMDb list if the first line of the file contains "Const".

The (?<=,) means that it must be preceded by a comma (the purpose is to make it ignore IMDb URLs in comments, which are unlikely to be preceded by commas).
So if I uploaded a list (csv) that had a single column of IMDB urls in there, your program would be able to read it?
Yes, I think so, but skip the first line because it assumes there are no IMDb URLs on the first line.

mjf314
Moderator
Posts: 10596
Joined: May 08, 2011
Contact:

#586

Post by mjf314 » December 31st, 2017, 12:51 am

IMDb List Helper and iCM Enhanced don't seem to be working for me (I'm not sure when they stopped working since I don't use them very often). Does anyone know what the problem might be?

I checked to make sure that Greasemonkey is enabled, and it is. I'm using Greasemonkey 4.1 and Firefox 56.0.2. I don't like the latest Firefox so I'd prefer not to update. Would reverting to an older version of Greasemonkey fix it?
Last edited by mjf314 on December 31st, 2017, 12:54 am, edited 1 time in total.


mjf314
Moderator
Posts: 10596
Joined: May 08, 2011
Contact:

#588

Post by mjf314 » December 31st, 2017, 1:20 am

iCM Enhanced works now, but not IMDb List Helper. Did the recent changes to IMDb cause it to stop working?

User avatar
PeacefulAnarchy
Moderator
Posts: 21707
Joined: May 08, 2011
Contact:

#589

Post by PeacefulAnarchy » December 31st, 2017, 1:24 am

Hmm, doesn't work for me either on firefox 57.02, script is version 2.4.1. I don't know.

User avatar
monk-time
Posts: 1334
Joined: Mar 23, 2015
Contact:

#590

Post by monk-time » December 31st, 2017, 1:37 am

mjf314 on Dec 30 2017, 05:51:58 PM wrote:I checked to make sure that Greasemonkey is enabled, and it is. I'm using Greasemonkey 4.1 and Firefox 56.0.2. I don't like the latest Firefox so I'd prefer not to update. Would reverting to an older version of Greasemonkey fix it?
Greasemonkey 4 made some backward incompatible changes, all scripts that have in the header any lines starting with "// @grant GM_" are not going to work unless the author includes a polyfill script. Same if they use page scripts. So stick to TM or Violentmonkey; note that only the latter is open source.

By the way, using FF56 is kind of dangerous. That version is not going to receive any fixes for security vulnerabilities. I understand that updating to FF57 is a world of pain for old Firefox users, but I recommend going through it anyway. Not updating is not safe.
ICM profile | My userscripts for IMDb/iCM | iCheckMovies Enhanced by themagician: call it direct, call it collect, but CALL IT TODAY!

User avatar
PeacefulAnarchy
Moderator
Posts: 21707
Joined: May 08, 2011
Contact:

#591

Post by PeacefulAnarchy » December 31st, 2017, 1:42 am

I updated and lost downloadthemall and I'm very sad. It's coming back but there's no ETA and while I don't use it a huge amount it saves a lot of time when I do use it.

User avatar
mightysparks
Site Admin
Posts: 29000
Joined: May 05, 2011
Location: Perth, WA, Australia
Contact:

#592

Post by mightysparks » December 31st, 2017, 5:15 am

Is there an updated version of that script thingy for IMDb that has the links on the side? Since IMDB has decided to get rid of reference view and replace it with some awful thing with useless information it hasn't worked. Also, it won't let me have the main title in the original language, it shows it in English. Goddamn IMDb, what the hell has happened to you.
"I do not always know what I want, but I do know what I don't want." - Stanley Kubrick

iCM | IMDb | LastFM | TSZDT

Image

User avatar
monk-time
Posts: 1334
Joined: Mar 23, 2015
Contact:

#593

Post by monk-time » December 31st, 2017, 5:27 am

mightysparks on Dec 30 2017, 10:15:47 PM wrote:Also, it won't let me have the main title in the original language, it shows it in English. Goddamn IMDb, what the hell has happened to you.
Site Settings → Update general site preferences:

Image

Image
ICM profile | My userscripts for IMDb/iCM | iCheckMovies Enhanced by themagician: call it direct, call it collect, but CALL IT TODAY!

User avatar
brokenface
Donator
Posts: 12862
Joined: Dec 29, 2011
Contact:

#594

Post by brokenface » December 31st, 2017, 7:38 pm

IMDB My Movies Enhancer seems to have been broken again by further changes - the button to re-run it no longer appears on the Lists page. Any chance there's been an update that I can't find?

User avatar
monk-time
Posts: 1334
Joined: Mar 23, 2015
Contact:

#595

Post by monk-time » December 31st, 2017, 7:59 pm

brokenface on Dec 31 2017, 12:38:55 PM wrote:IMDB My Movies Enhancer seems to have been broken again by further changes - the button to re-run it no longer appears on the Lists page. Any chance there's been an update that I can't find?
https://openuserjs.org/scripts/AltoRetr ... s_enhancer

This is the latest version from the author, should work on the new layout. Also see the discussion a dozen of posts above.
ICM profile | My userscripts for IMDb/iCM | iCheckMovies Enhanced by themagician: call it direct, call it collect, but CALL IT TODAY!

User avatar
Lonewolf2003
Donator
Posts: 7281
Joined: Dec 29, 2012
Contact:

#596

Post by Lonewolf2003 » January 1st, 2018, 5:26 pm

Sorry with all these post I might have missed it. But the listdownloader isn't working anymore. It only downloads icm-lists. Is there a updated version (since the imdb list screw-up updates) that works, if so what is it?

Or maybe the cookie has changed? But I forgot how to see a cookie (in Chrome). Used to have an add-on for that, but that seems gone too.

User avatar
jvv
Donator
Posts: 8082
Joined: May 28, 2011
Location: Netherlands
Contact:

#597

Post by jvv » January 1st, 2018, 5:39 pm

mightysparks on Dec 30 2017, 10:15:47 PM wrote:Is there an updated version of that script thingy for IMDb that has the links on the side?
I would like to know this as well.

User avatar
joachimt
Donator
Posts: 27830
Joined: Feb 16, 2012
Location: Netherlands
Contact:

#598

Post by joachimt » January 1st, 2018, 6:37 pm

Lonewolf2003 on Jan 1 2018, 10:26:04 AM wrote:Sorry with all these post I might have missed it. But the listdownloader isn't working anymore. It only downloads icm-lists. Is there a updated version (since the imdb list screw-up updates) that works, if so what is it?

Or maybe the cookie has changed? But I forgot how to see a cookie (in Chrome). Used to have an add-on for that, but that seems gone too.
mjf314 updated it, because of IMDb-changes.

http://www.icmforum.com/single/?p=10114636&t=7272196
ICM-profile
Fergenaprido: "I find your OCD to be adorable, J"

User avatar
monk-time
Posts: 1334
Joined: Mar 23, 2015
Contact:

#599

Post by monk-time » January 1st, 2018, 6:42 pm

jvv on Jan 1 2018, 10:39:06 AM wrote:
mightysparks on Dec 30 2017, 10:15:47 PM wrote:Is there an updated version of that script thingy for IMDb that has the links on the side?
I would like to know this as well.
That's themagician's IMDb details page links.
ICM profile | My userscripts for IMDb/iCM | iCheckMovies Enhanced by themagician: call it direct, call it collect, but CALL IT TODAY!

User avatar
Lonewolf2003
Donator
Posts: 7281
Joined: Dec 29, 2012
Contact:

#600

Post by Lonewolf2003 » January 1st, 2018, 6:44 pm

joachimt on Jan 1 2018, 11:37:56 AM wrote:
Lonewolf2003 on Jan 1 2018, 10:26:04 AM wrote:Sorry with all these post I might have missed it. But the listdownloader isn't working anymore. It only downloads icm-lists. Is there a updated version (since the imdb list screw-up updates) that works, if so what is it?

Or maybe the cookie has changed? But I forgot how to see a cookie (in Chrome). Used to have an add-on for that, but that seems gone too.
mjf314 updated it, because of IMDb-changes.

http://www.icmforum.com/single/?p=10114636&t=7272196
That's the counter, I meant the one that autodownloads all the list. Or does this does that as well?

Anyhow I already downloaded (almost all) list for Korea manually.

Post Reply