Yeah, I’ve had a look. There were definitely a couple of things that could have affected some of the Latest Topics code. The links between topics and tags was a big fragmented, which I’ve sorted. I also restarted the site a bit, and we’re a bit faster than we were, still nowhere as fast as I would like.
Yep definitely quicker response now. I have a very quick broadband connection (150mb/sec) and clicking on “latest” was taking 15 sec + to load earlier. It’s about 5 secs now.
Small update. A number of changes have been made to address things.
First off, we noticed that the “Latest By” link was not doing any real paging. It’d load the entire thread, which on something like Optimus Trousers, which is replete with content, that was taking a long fucking time.
It has been disabled. It was never really required after the visit code went in, and while it probably suits the kind of threads that develop on Q+A sites, it’s no good for the sort of long-running threads we have here.
Now I don’t know how many people were using Latest By, but just a couple hitting big threads could have had big implications for other people using the site. Seems to be running loads faster now.
Whilst Optimus trousers isn’t the problem, per se, the fact that Optimus trousers’ thread is now so long is exacerbating the performance problems we’re having. But the same is true of any long threads we have.
The performance of the site will still be poor on the home page until the Optimus trousers thread has made its way off the home page. So get posting and push it down the latest thread list, at which point things should speed up.
Papster is working on a couple of options to fix this and will tell me when I can pull my trousers up again *.
* Double entendre police to the Site Performance thread please!
It’s got bugger all to do with size or indexing; just some curious design choices that we’ve exposed with the way we run this site.
Remember that this is basically an open source take on StackOverflow. At its heart, it is more a Q & A site than an actual discussion forum. I think the original devs never expected huge threads, which accounts for most of the slowness.
They’ve basically coded it so that it makes logical sense from a developer’s point of view, which is not always what you want when performance is your key goal.
An example. That Latest Topic page currently works by:-
Loading the n most recent topics
Loading all posts attached to those topics
Loading all votes attached to those posts
Step by step, it makes utter sense, but there is a chasm between what we actually need and what we pull back. The only reason we’re pulling votes is to get an up/down count. That is one of the big reasons it pulls all the posts.
I’m working on replacing the main pages that show topics with alternatives that’ll load a ton faster. I’ve done this by considering only the fields we need to load that page. I would expect most performance problems to completely fall away at this point.
No offence detected, BBB. I was merely dumping my investigation somewhere.
The leaderboard vote count incorporates everything. I was just explaining that the code was going a really slow way about getting the counts that are visible on the home page.
As papster described above, to display the home page (latest posts), the code grabs the latest n threads and then grabs all of the posts for all of those threads.
This is done just to display the list of threads where the posts inside the thread are not even visible.
This happens every time someone refreshes the home page. Add Optimus trousers’ mammoth ~300 page thread into that and the server is thrashing on every F5.
The reason seems to be down to a desire show summary stats next to the thread that come from the posts.
It’s either very poor design, or as papster suggested, we’re using the forum in a way that was not intended/envisaged.
Re: horses for courses…Yep. I was just yanking your tail.