Welcome to the ICM Forum. If you have an account but have trouble logging in, or have other questions, see THIS THREAD.
NOTE: Board emails should be working again. Information on forum upgrade and style issues.
Podcast: Talking Images (Episode 22 released November 17th * EXCLUSIVE * We Are Mentioned in a Book!!! Interview with Mary Guillermin on Rapture, JG & More)
Polls: Directors (Waiting for results), 1929 (Results), Directorial Debut Features (Mar 12th), DtC - Nominations (Mar 20th)
Challenges: Experimental/Avant Garde, Benelux, Run the Director
Film of the Week: Daisan no kagemusha, March nominations (Feb 26th)
NOTE: Board emails should be working again. Information on forum upgrade and style issues.
Podcast: Talking Images (Episode 22 released November 17th * EXCLUSIVE * We Are Mentioned in a Book!!! Interview with Mary Guillermin on Rapture, JG & More)
Polls: Directors (Waiting for results), 1929 (Results), Directorial Debut Features (Mar 12th), DtC - Nominations (Mar 20th)
Challenges: Experimental/Avant Garde, Benelux, Run the Director
Film of the Week: Daisan no kagemusha, March nominations (Feb 26th)
icm is down
-
- Posts: 498
- Joined: April 1st, 2012, 6:00 am
- Contact:
I'm getting the 502 message now, but wasn't 15 minutes ago
Worked perfect for me all day long. Weird.
It's probably only for an hour or so, hopefully they are trying to implement the easy adoption process.
-
- Posts: 498
- Joined: April 1st, 2012, 6:00 am
- Contact:
or kick off some cheaters
well its back nowmimurph1 on Sep 18 2013, 11:58:50 AM wrote:major problems
Turning over a new leaf 

- Armoreska
- Posts: 12833
- Joined: November 1st, 2012, 6:00 am
- Location: Ukraine, former Free Territory
- Contact:
down again or "temporarily inaccessible"
Last edited by Armoreska on September 19th, 2013, 9:32 pm, edited 1 time in total.
he or A. or Armo or any

currently working towards a vegan/free world + thru such film lists: GODARD,
imaginary awards | youtube channels | complaint lounge | explain how big a fan of slavery you are here, ..viewtopic.php?f=12&t=1535 and here: ..viewtopic.php?f=12&t=4484

currently working towards a vegan/free world + thru such film lists: GODARD,
Spoiler
ANARCHISTS, 2010s bests, Yasujiro Ozu, Visual Effects nominees, kid-related stuff, great animes (mini-serie or feature), very 80s movies, 17+ sci-fi lists on watchlist, ENVIRO, remarkable Silent Films and Pre-Code (exploring 1925 atm) and every shorts and docu list I'm aware of and
/forum.icmforum.com/viewtopic.php?f=1&t=1434
and "Gordon" Liu Chia-Hui/Liu Chia-Liang and Yuen Woo-ping and "Sammo" Hung Kam-bo
/forum.icmforum.com/viewtopic.php?f=1&t=1434
and "Gordon" Liu Chia-Hui/Liu Chia-Liang and Yuen Woo-ping and "Sammo" Hung Kam-bo
We're up again... for now. Our database server can't cope with the growth of the site. We're planning an upgrade soon. Sorry about this.
I guess that means you need an official forum for days like these. 

Thanks for the update, you guys are doing a terrific job!Marijn on Sep 19 2013, 03:58:21 PM wrote:We're up again... for now. Our database server can't cope with the growth of the site. We're planning an upgrade soon. Sorry about this.
Which part of iCM puts the most strain on the servers? Is it neighbor compatibility, or something else? Maybe you could reduce how often neighbor compatibility gets calculated, or maybe only calculate it for active users?
Or, if you're in the mood for a more radical change, maybe just get rid of the neighbor system and replace it with something better - a recommendation database. I'm not sure if that would put more or less strain on the servers, but I'm guessing less because you wouldn't have to recalculate so many scores on a regular basis.
Or, if you're in the mood for a more radical change, maybe just get rid of the neighbor system and replace it with something better - a recommendation database. I'm not sure if that would put more or less strain on the servers, but I'm guessing less because you wouldn't have to recalculate so many scores on a regular basis.
Last edited by mjf314 on September 19th, 2013, 10:11 pm, edited 1 time in total.
-
- Posts: 11744
- Joined: May 29th, 2011, 6:00 am
- Contact:
Maybe if you're good friends with the iCM server you can just have a chat and find out what's wrong?mjf314 on Sep 19 2013, 04:07:47 PM wrote:Which part of iCM puts the most strain on the servers? Is it neighbor compatibility, or something else? Maybe you could reduce how often neighbor compatibility gets calculated, or maybe only calculate it for active users?
Or, if you're in the mood for a more radical change, maybe just get rid of the neighbor system and replace it with something better - a recommendation database. I'm not sure if that would put more or less strain on the servers, but I'm guessing less because you wouldn't have to recalculate so many scores on a regular basis.

- St. Gloede
- Moderator
- Posts: 12045
- Joined: May 6th, 2011, 6:00 am
- Contact:
biglolLocal Hero -- aka MestnyiGeroi on Sep 19 2013, 04:15:35 PM wrote:Maybe if you're good friends with the iCM server you can just have a chat and find out what's wrong?mjf314 on Sep 19 2013, 04:07:47 PM wrote:Which part of iCM puts the most strain on the servers? Is it neighbor compatibility, or something else? Maybe you could reduce how often neighbor compatibility gets calculated, or maybe only calculate it for active users?
Or, if you're in the mood for a more radical change, maybe just get rid of the neighbor system and replace it with something better - a recommendation database. I'm not sure if that would put more or less strain on the servers, but I'm guessing less because you wouldn't have to recalculate so many scores on a regular basis.
![]()
- mightysparks
- Site Admin
- Posts: 31319
- Joined: May 5th, 2011, 6:00 am
- Location: Perth, WA, Australia
- Contact:
Local Hero -- aka MestnyiGeroi on Sep 19 2013, 04:15:35 PM wrote:Maybe if you're good friends with the iCM server you can just have a chat and find out what's wrong?mjf314 on Sep 19 2013, 04:07:47 PM wrote:Which part of iCM puts the most strain on the servers? Is it neighbor compatibility, or something else? Maybe you could reduce how often neighbor compatibility gets calculated, or maybe only calculate it for active users?
Or, if you're in the mood for a more radical change, maybe just get rid of the neighbor system and replace it with something better - a recommendation database. I'm not sure if that would put more or less strain on the servers, but I'm guessing less because you wouldn't have to recalculate so many scores on a regular basis.
![]()

mightysparks on Sep 19 2013, 04:17:43 PM wrote:Local Hero -- aka MestnyiGeroi on Sep 19 2013, 04:15:35 PM wrote:Maybe if you're good friends with the iCM server you can just have a chat and find out what's wrong?mjf314 on Sep 19 2013, 04:07:47 PM wrote:Which part of iCM puts the most strain on the servers? Is it neighbor compatibility, or something else? Maybe you could reduce how often neighbor compatibility gets calculated, or maybe only calculate it for active users?
Or, if you're in the mood for a more radical change, maybe just get rid of the neighbor system and replace it with something better - a recommendation database. I'm not sure if that would put more or less strain on the servers, but I'm guessing less because you wouldn't have to recalculate so many scores on a regular basis.
![]()

You guys should remove this Delluc profile, that should make a big difference. 

Hey eezee does it!Gershwin on Sep 19 2013, 04:28:22 PM wrote:You guys should remove this Delluc profile, that should make a big difference.![]()
No, it's not only the neighbors compatibility (we already calculate neighbors once a week only for active users). It's just the overall size of the database combined with our database design. MySQL just isn't made for the things we do with it. We need to move certain data to other database systems (such as Redis and Neo4j) and use queuing (RabbitMQ) to update everything in the background when a user interacts with the site.mjf314 on Sep 19 2013, 04:07:47 PM wrote:Which part of iCM puts the most strain on the servers? Is it neighbor compatibility, or something else? Maybe you could reduce how often neighbor compatibility gets calculated, or maybe only calculate it for active users?
Or, if you're in the mood for a more radical change, maybe just get rid of the neighbor system and replace it with something better - a recommendation database. I'm not sure if that would put more or less strain on the servers, but I'm guessing less because you wouldn't have to recalculate so many scores on a regular basis.
The biggest problems at the moment are with:
1. Messages (after toplist update, awards update or blog post)
2. Daily update of the table used for award and neigbor calculation (basically a truncate and reinsert)
3. Neighbor calculation
4. Top user calculation
We've already built a proof of concept with queuing so we know it's working, but it's just a lot of work to implement it everywhere. We're working as fast as we can, but as some of you know, we have to work on the site in our spare time. We all have full time jobs.
Anyways, thanks for using the site. You're all awesome!
Last edited by Marijn on September 23rd, 2013, 3:57 pm, edited 1 time in total.
I've never understood the neighbor compatibility thing. I have yet to find anyone whose compatibility with me is anything but low (except for my buddy Cinephage, who is VERY low).

But couldn't all these activities you mention be made less frequent in the meantime (until you move to new technology)?Marijn on Sep 23 2013, 09:55:47 AM wrote:No, it's not only the neighbors compatibility (we already calculate neighbors once a week only for active users). It's just the overall size of the database combined with our database design. MySQL just isn't made for the things we do with it. We need to move certain data to other database systems (such as Redis and Neo4j) and use queuing (RabbitMQ) to update everything in the background when a user interacts with the site.mjf314 on Sep 19 2013, 04:07:47 PM wrote:Which part of iCM puts the most strain on the servers? Is it neighbor compatibility, or something else? Maybe you could reduce how often neighbor compatibility gets calculated, or maybe only calculate it for active users?
Or, if you're in the mood for a more radical change, maybe just get rid of the neighbor system and replace it with something better - a recommendation database. I'm not sure if that would put more or less strain on the servers, but I'm guessing less because you wouldn't have to recalculate so many scores on a regular basis.
The biggest problems at the moment are with:
1. Messages (after toplist update, awards update or blog post)
2. Daily update of the table used for award and neigbor calculation (basically a truncate and reinsert)
3. Neighbor calculation
4. Top user calculation
We've already built a proof of concept with queuing so we know it's working, but it's just a lot of work to implement it everywhere. We're working as fast as we can, but as some of you know, we have to work on the site in our spare time. We all have full time jobs.
Anyways, thanks for using the site. You're all awesome!
I for one really don't care about real-time updates to any of those things (in fact I wouldn't miss if the neighbors thing is completely removed), I'm pretty sure others can wait a bit as well if this is causing such strain on the servers.
Just my 2¢

Last edited by bal3x on September 23rd, 2013, 4:07 pm, edited 1 time in total.
So are you. Thanks for all the time you spend on our hobby. 

- PeacefulAnarchy
- Moderator
- Posts: 25938
- Joined: May 8th, 2011, 6:00 am
- Contact:
None of those things happen in real time right now. I wouldn't want the award and top user calculations to be less frequent. I could stand to not have the biweekly update messages if there was a permanent place on the site where that info could be easily found, same with award updates. I like the messages but they're not strictly necessary. I don't use the neighbors at all.bal3x on Sep 23 2013, 10:06:20 AM wrote:But couldn't all these activities you mention be made less frequent in the meantime (until you move to new technology)?
I for one really don't care about real-time updates to any of those things (in fact I wouldn't miss if the neighbors thing is completely removed), I'm pretty sure others can wait a bit as well if this is causing such strain on the servers.
Last edited by PeacefulAnarchy on September 23rd, 2013, 4:21 pm, edited 1 time in total.
Thanks for all your work!
Yes, I know that, real-time was a wrong term I used, but the idea is to have less frequent updates until new technology is implemented. Just a temporary solution. If less frequent updates means faster site and less down-times I'm certainly willing to accept that compromise. Anyways, thanks for all the hard work, Guys, I'm sure you will arrive at the best solution!PeacefulAnarchy on Sep 23 2013, 10:20:44 AM wrote:None of those things happen in real time right now. I wouldn't want the award and top user calculations to be less frequent. I could stand to not have the biweekly update messages if there was a permanent place on the site where that info could be easily found, same with award updates. I like the messages but they're not strictly necessary. I don't use the neighbors at all.bal3x on Sep 23 2013, 10:06:20 AM wrote:But couldn't all these activities you mention be made less frequent in the meantime (until you move to new technology)?
I for one really don't care about real-time updates to any of those things (in fact I wouldn't miss if the neighbors thing is completely removed), I'm pretty sure others can wait a bit as well if this is causing such strain on the servers.

Maybe the awards could be sent out only once a week (temporarily?). I know this may not be a popular solution. The once a day calculation of checks is good, though. Any less would be a bit frustrating.Marijn on Sep 23 2013, 09:55:47 AM wrote:No, it's not only the neighbors compatibility (we already calculate neighbors once a week only for active users). It's just the overall size of the database combined with our database design. MySQL just isn't made for the things we do with it. We need to move certain data to other database systems (such as Redis and Neo4j) and use queuing (RabbitMQ) to update everything in the background when a user interacts with the site.mjf314 on Sep 19 2013, 04:07:47 PM wrote:Which part of iCM puts the most strain on the servers? Is it neighbor compatibility, or something else? Maybe you could reduce how often neighbor compatibility gets calculated, or maybe only calculate it for active users?
Or, if you're in the mood for a more radical change, maybe just get rid of the neighbor system and replace it with something better - a recommendation database. I'm not sure if that would put more or less strain on the servers, but I'm guessing less because you wouldn't have to recalculate so many scores on a regular basis.
The biggest problems at the moment are with:
1. Messages (after toplist update, awards update or blog post)
2. Daily update of the table used for award and neigbor calculation (basically a truncate and reinsert)
3. Neighbor calculation
4. Top user calculation
We've already built a proof of concept with queuing so we know it's working, but it's just a lot of work to implement it everywhere. We're working as fast as we can, but as some of you know, we have to work on the site in our spare time. We all have full time jobs.
Anyways, thanks for using the site. You're all awesome!
Maybe update the neighbors once a month? I'm sure people hardly ever look at that.
Or eliminate them completely. I don't think I'll even notice if they're gone.allisoncm on Sep 23 2013, 12:32:28 PM wrote:Maybe update the neighbors once a month? I'm sure people hardly ever look at that.

I agree. Some users like this features, but I rarely use it.frbrown on Sep 23 2013, 01:03:28 PM wrote:Or eliminate them completely. I don't think I'll even notice if they're gone.allisoncm on Sep 23 2013, 12:32:28 PM wrote:Maybe update the neighbors once a month? I'm sure people hardly ever look at that.
1. I posted a suggestion regarding this issue on Trello several months ago:Marijn on Sep 23 2013, 09:55:47 AM wrote:1. Messages (after toplist update, awards update or blog post)
2. Daily update of the table used for award and neigbor calculation (basically a truncate and reinsert)
3. Neighbor calculation
4. Top user calculation
https://trello.com/c/OaxSqtVP/567-membe ... ts-anymore
I might be wrong because I'm not very knowledgeable about this kind of stuff, but I think changing 140,000 boolean variables (and then displaying a notification that links to a single page the next time the user visits the site) would be much less work than sending 140,000 messages. I know you mentioned a notification system, so maybe you were already planning to do something similar to this.
4.
Edit: Actually forget what I just said. Sorting 137759 numbers will get done quickly regardless of algorithm, so that's the easy part. I guess the hard part is calculating how many official checks each user has? Unfortunately I don't have any suggestions for how to improve that part.
Last edited by mjf314 on September 24th, 2013, 3:55 am, edited 1 time in total.
Anyone else having difficulties as of right now?