You've likely been wondering why the site has been slow the past week. I've been wondering the same.

The Issue

Sluggish performance on the page displaying your past giveaway entries, and more recently, all pages. This seemed to occur primarily during peak hours, but also at what appeared to be random, during off-peak hours as well.

Why Is This Happening?

In short, we don't have enough ram to store our active dataset and indexes, so the site needs to occasionally read from disk. For example, a user attempts to view their giveaway entries from the past. Our server can't hold all of our giveaway entry information in our limited amount of ram, so this information would be read from disk. Scanning hundreds of millions of rows on disk to find the related data for a user is an incredibly slow process, and it might take 30+ seconds, when it should be taking 0.02 seconds if the data was available in ram. This explains why the page displaying your past giveaway entries would take a long time to load if you haven't accessed it in a while, since it would be reading from disk instead of ram.

Now, what about other pages, why has everything been slow lately? Well, the disks are so busy working overtime, trying to read the data for the above user, they're not available for other users browsing the site, and these users are left waiting, and waiting, and potentially timing out. If we had more ram available, the original user would be reading their data directly from ram, and the disks would be free for writes.

Solution

We need to download more ram. Our database server runs 24gb of ram at the moment, and that's far too low for our 60gb of data and indexes. I received a price today for upgrading to 128gb of ram, and we'll soon be going through with those upgrades. However, we need a short term solution so users can continue using the site before that happens. I ended up disabling the page for viewing your past giveaway entries. This keeps the hard drives from being a bottleneck and spending all of their time reading that information for a small number of users, when they need to focus on writes to the database for everyone else. It's peak hours right now, and you should notice the site being quite responsive, so I think this solution will work well until the ram upgrade.

Edit: The entered giveaway page will now show 100 of your past entries, in descending order. For performance reasons, this is the only way the results can be ordered and displayed until the server is upgraded, and full functionality can be restored.

9 years ago*

Comment has been collapsed.

Thanks for all you do to keep the site running smoothly! If the problem was the View Entered (giveaways), could comment notifications be re-enabled prior to the RAM upgrade?

9 years ago
Permalink

Comment has been collapsed.

Don't know what use, right now new interesting GAs is created and points is still 0, if GA run only for 1hr, then need to remove entry from something else=view page Entered-this is disabled, any solution for this?

9 years ago
Permalink

Comment has been collapsed.

You will have to browse through public GAs to leave ones you're less interested in. And now when site is far better responsive it shoudn't be a problem.

9 years ago
Permalink

Comment has been collapsed.

you think so? all 60 pages (for me) don't think it is fast to find less interested, something else solution?
new solution, remove from interesting GAs but from which last for 2-3 weeks like this will browse less pages...

9 years ago*
Permalink

Comment has been collapsed.

There isn't any other solution. Only other thing you can do is press "last" button and leave GAs from last few pages (which will be open for another 1 - 2 weeks) and re-enter them later.

9 years ago
Permalink

Comment has been collapsed.

You are awesome! Really!

9 years ago
Permalink

Comment has been collapsed.

Thanks cg, great work ;)
Keep it up!

9 years ago
Permalink

Comment has been collapsed.

Thanks CG! You are most definitely thebomb.com!

9 years ago
Permalink

Comment has been collapsed.

Deleted

This comment was deleted 5 years ago.

9 years ago
Permalink

Comment has been collapsed.

I tried downloading all the RAM, but I ran out of disk space.

9 years ago
Permalink

Comment has been collapsed.

IIRC there is another site that will let you download more GB of disk space. Deleting the system32 folder frees up a lot of space too.
;)

9 years ago
Permalink

Comment has been collapsed.

Deleted

This comment was deleted 5 years ago.

9 years ago
Permalink

Comment has been collapsed.

Thank you for your efforts!

But as some other users I miss the "entered"-Section. Is there a possibility that you could make the active GAs available while deactivating the finished GAs? Or would that make thigs worse again?

Just that you get me right, I'm just curious. I think nobody should take it for granted that you invest in this page and make such a community even possible without charging anything! So thank you!

9 years ago
Permalink

Comment has been collapsed.

Deleted

This comment was deleted 6 years ago.

9 years ago
Permalink

Comment has been collapsed.

Coolio!appreciated. . .

9 years ago
Permalink

Comment has been collapsed.

PS: Maybe you should change the information in the "entered"-section. It says: "Page disabled for a few moments." and since this is until further notice it may be wise to write so (or you may be flooded with support tickets and users opening new threads, why it doesn't work for them)

9 years ago
Permalink

Comment has been collapsed.

Thanks for the update cg :)

9 years ago
Permalink

Comment has been collapsed.

Since the beginning of times, people have been in need of more and more ram.

Thanks cg for keeping on top of this!
BTW, I found a bug... the giveaway entered page won't load anymore. :P

View attached image.
9 years ago
Permalink

Comment has been collapsed.

I ended up disabling the page for viewing your past giveaway entries.

Do you even read, mate? :P

9 years ago
Permalink

Comment has been collapsed.

View attached image.
9 years ago
Permalink

Comment has been collapsed.

I don't get it. :(

9 years ago
Permalink

Comment has been collapsed.

How about make crowd-funding for buying ram? If every user spend just 1$ it would be a lot!

9 years ago
Permalink

Comment has been collapsed.

Thanks cg.

9 years ago
Permalink

Comment has been collapsed.

Perhaps a running a daily cron job to delete inbox messages older than 30 or 60 days might help to keep the database size at a more reasonable level as well. Just an idea.

9 years ago
Permalink

Comment has been collapsed.

thanks cg!

just don't disable my spam, it's the only thing i can offer to the site :3

9 years ago
Permalink

Comment has been collapsed.

Oh god, but I enter giveaways in advance. How am I now going to magically gave more points reapear? :O
But anything for the better site performance I guess :(

9 years ago
Permalink

Comment has been collapsed.

Reminiscing about past giveaways is of course non-essential, but access to current/open entries is kinda vital (I'm thinking of withdrawing identical entries after a win).

Would it be possible to have two different pages for two different queries? As in "past entries" versus "open entries"? Would the db structure allow the latter to be performant?

Edit: "Limited number of results". I'm ok with that.

9 years ago*
Permalink

Comment has been collapsed.

+1 for having two different pages for two different queries.

9 years ago
Permalink

Comment has been collapsed.

+1.

9 years ago
Permalink

Comment has been collapsed.

The solution, while waiting for the ram upgrade, it's working great! Finally the website is fast and responsive!

9 years ago
Permalink

Comment has been collapsed.

Thanks a lot, cg :)

9 years ago
Permalink

Comment has been collapsed.

Totally reading through hoping to get the skinny on what issues they were having... then:

"We need to download more ram."

Perfect straight-line delivery, I laughed out loud....well played.

Hope the upgrades go smoothly for you guys, cheers! And thanks for the hard work you guys do behind the scenes

9 years ago
Permalink

Comment has been collapsed.

To be honest I'm not really sure I get where the value is in preserving the database of "stuff you didn't win" but good luck with your upgrades and thanks for keeping us informed.

9 years ago
Permalink

Comment has been collapsed.

It has a number of potential uses. First off, we need it for rerolling past giveaways, and occasionally we need to verify information in user tickets, to know who entered a giveaway, and when. Users also want to occasionally search for a giveaway they previously entered, such as an invite only one, where they were having a conversation. It also has potential for fun stats, showing which games you entered to win the most, estimated wins to display if you're lucky or unlucky, and more. It would be unfortunate to not have that data readily available, simply to save a little on costs.

9 years ago
Permalink

Comment has been collapsed.

Wouldn't keeping closed giveaways on a separate page ease the server load though? I think that people care more about active giveaways than ones that are closed or ones that ended a year ago. Having them on separate pages should help relieve the load a bit one would think since the closed ones wouldn't have to be accessed as much.

9 years ago
Permalink

Comment has been collapsed.

Sounds like a good solution.

9 years ago
Permalink

Comment has been collapsed.

I agree with what bmh67wa has written because I also go to the entered giveaway page only to check on my entered giveaways that have not yet finished. Maybe this suggestion can reduce the load on the server.

9 years ago
Permalink

Comment has been collapsed.

I imagine the other comments here about splitting the pages doesn't fully solve your problem unless you make a data model change to physically split GAs that have ended to a different table, and then you have the overhead of moving them.

If you take the query you're using right now with TOP 100 and add a filter to only return active GAs, does that kill the system again? Because that would help us out a lot I think. Right now I'm really only getting about 30 active GAs on the screen because it's filled with closed ones, and I have no idea about the ones I entered days ago that may or may not still be active.

9 years ago
Permalink

Comment has been collapsed.

And there it is again: Thank you so much for reactivating the "Entered"-Page. This is great service and helps a lot. Went straight through my GAs and deleted all the Dr. Who-GA's I entered in before buying the bundle myself! :)

9 years ago
Permalink

Comment has been collapsed.

Thanks cg! you rock at keeping steamgifts the slickest of the click!
Thanks for all you do for that matter!
The world would probably crumble without you. :3

9 years ago
Permalink

Comment has been collapsed.

Good job investigating.

However, I am wondering why does/did viewing your entered giveaways page also load any closed giveaways by default? This doesn't make sense to me because I feel like the primary reason that people go to that page is to get a listing of currently open giveaways that they have entered and/or to remove entries. I know I go there mostly to shuffle points around. Occasionally I may want look for a giveaway that has closed but it doesn't make sense to me to devote resources to loading them by default.

9 years ago*
Permalink

Comment has been collapsed.

I agree with this. Some giveaways that I joined were a couple of weeks long and now are not showing on the list. If I wanted to leave these giveaways for whatever reason I would have to scroll through the giveaway pages to find them again. If the closed giveaways were separate it would reduce the server load too since that page wouldn't be accessed as much as the page with open giveaways.

9 years ago
Permalink

Comment has been collapsed.

Good work cg! Thanks!

9 years ago
Permalink

Comment has been collapsed.

Closed 9 years ago by cg.