I know there's already an old thread here requesting API for SG, but it's 6 years old so I decided to make a new one. I'll try to cover the points from the old thread and any new ones I can think of.

I believe SG needs to offer an API. There are several tools, extensions & userscripts that scrape the website for the desired information, which is "bad" for both the tools and SG. An official API would benefit both. I'll try covering all the points I can think of.

What should the API offer:

  • Tokenized API for registered members only. Will also make it easier to make sure that only the info that the user should have access to is provided by the API. Also, can be used to limit the number of API hits by the user.
  • List all GAs, completed, ongoing and not-started GAs, all of them (but only those that the user have access to, that means anything but invite only GAs I believe, (users can see WL-only GAs too, just only access it's page/enter them)). The list should include these: GA Code/ID(the 5 letter GA code), Steam Link/ID, Game Name (optional), Start Time, End Time, Creator Username/Steam ID, GA Visibility (Groups / WL / Public, Level & Region restrictions), number of copies. Many of these could be omitted in the list and instead be provided in the below request. Filtering by start/end date, number of copies, game/steam ID, user's wishlist, group, etc should be supported, same as how search and the GA list on other "special" pages work. This is implemented to some extend, but a lot can be imporved. See more here.
  • Fetching info about a particular GA by it's GA Code. This should include all of the above, along with all the entries (username + timestamp of the entry), Winner(s) Usernames/SteamIDs after they have provided feedback with the feedback they provided. It's could also provide number of comments, just the total number of comments.
  • List of groups the user is a part of (same as https://www.steamgifts.com/account/steam/groups), with the group ID and the name. Filters could be provided as well (group name, number of users, etc).
  • Retrieving group info by it's ID. All the publicly visible information about the groups, like name, steam link, First GA, Last GA, num of GAs, list of GAs, list of users, group wishlist, etc. I don't think there's anything here that might need to be omitted.
  • User info by steam ID and SG username. The user info should include all the basic stats that is visible on the user page (role, last online, registered, num of comments, etc), list of created and won GAs code (just the GA codes is enough, as we can already fetch info about GAs from the 3nd point). Also, this should include the current status of the user (active, account suspended, permanently banned, etc).

What should the API NOT offer:

  • The API data should be read-only. That means it should NOT be possible to enter GAs, create GAs, make comments, or make any kind of changes.
  • I am a bit unsure about this, but I think the GA descriptions and GA comment contents should not be accessible by API.
  • Access the won GAs' keys/gift links. Even though the user is authenticated via the API token, still access to even revealed keys should NOT be available imo.
  • Access to tickets/support system of any kind. I believe they don't require API. Though, even if there were an API for it, I can't think of an use for it.
  • Access to discussions of any kind. They are like forum, and hence don't require API imo, and they can be too much data. Also, discussion usually contain links to all the invite only GAs, or puzzles and stuff. So, not providing access to it might be better (even though the bots can still crawl the website and get all the info from there directly. They don't even need to log in for accessing the discussions).

How it will benefit everyone (or at least tools developers and tools users):

  • The tools developers will no longer have to scrape the website. Tools like SGTools and ESGST are very common, and cause a lot of load on the website. And API could make the tools development easier.
  • This is reduce web scraping to a great degree (it won't eliminate it completely, specially due to bot accounts). Web scraping puts a lot more load on the server than APIs (even if just the static HTML is used for scrapping, without the images and external references).
  • Users of multiple tools like ESGST, "do you even play bro", etc, could unknowing exhibiting DOS attack like behavior from the server's perspective. Though most users should not hit the hit limit easily, but I am sure at least might occasionally hit the limit and get the 429 error. An API could prevent, or reduce this.

Some concerns people might have:

  • This might create extra maintenance load for cg: The same API can be used for the normal website too. This way cg (future SG devs) will not have to maintain data access in two places. Sure, there will be some extra maintenance workload if and when updating things, but reusing the API will reduce it quite significantly.
  • Bots/hackers/cheaters will be able to do the "evil" stuff more easily: They can still do it almost just as easily via web scraping, user scripts, etc. And that stresses the server more than APIs. APIs or the lack of them are not gonna make any difference in preventing such users from doing their thing.
  • People can make their own GUI/interface/application/etc and use that to access the site, which means no ad revenue for SG: Well, such tools can still be created, and web scaping doesn't generate ad revenue either. However, API might reduce the server load, and could potentially reduce the server cost. Besides, since the API will provide only read access to the data, people will still have to visit the website to actually enter GAs, create them, comment, etc).

Why am I so concerned whether SG has an API or not:
I don't how to put it nicely, but SG has almost no implementations to prevent abusers (multi-account users, re-gifters, etc). All such cases are handled manually. And I as unfortunate enough to come across some of them recently. Well, I had some free time, and I like wasting my time on useless things, so I decided to make some tools to try discover such users. Even though I managed to make the tools I wanted, but during the whole development phase I felt the lack of an official API.
Also, it dawned on me that some of the great tools like SGTools, and the super popular tools like ESGST must be scrapping the website too. And unlike my tool, they probably are being used by 1000s of people, and they run every time these users load a SG page. Which can't be good for SG. Update: Actually, some of the features in these tools might be using the JSON support found here. So, it's possible they might be scrapping only partially.
Also, a quick google search indicated there have been some failed attempts to make unofficial APIs for SG by scrapping the website. Though I believe they are abandoned project, but they are just equally bad for the web server.
So, imo, an official API could help all who use, develop or are impacted by such tools, while lowering the server load.

Anyway, thanks to anyone who read this. And feel free to share your thoughts.

Even though I have very little hope that we will ever get an API, I still had to create this thread, I guess to blow off some steam. Actually, I know we are not going to get an API anytime soon.


Thanks to NoSenses to pointing out to this thread.
Seems we already sort of the API, with at least few of the things I am asking for below. However, it seems to still be missing a lot of things from below.

2 years ago*

Comment has been collapsed.

I don't believe we will ever see any radical improvements on SG, but I hope we will :) And having API is critical to let others implement what admin doesn't want to do by himself.

PS: Tiny giveaway (lvl4+ till 12 of August) to support the cause
PPS: Tell me more about your tool, sounds interesting.

2 years ago*
Permalink

Comment has been collapsed.

I agree. I have almost given up hope on seeing any improvements. And I have been feeling a bit frustrated lately.
Yeah, I agree. people are already trying to do that. If the admin provided an API, people would be able to it just a little more easily.

Thanks for the "support". I forgot to add one to the post (not that I had plans to, but it's tradition here).

Well, the tools basically scraps some data for me and generates some very basic overview reports, that I then analyze. The analysis is most manual since otherwise there would be a lot of false positive. I have used this technique just twice, so I believe I'll be tweaking it as I continue using it.
So, basically, it's nothing fancy. It's just glorified web scrapper.

2 years ago
Permalink

Comment has been collapsed.

Bump because I agree!

2 years ago
Permalink

Comment has been collapsed.

It is a cool idea, yes, does it help extension developers? yes, but it's something we probably wouldn't see implemented soon, as it'd require to change a bit for it to behave nice, plus the server load will probably be higher plus costs (don't forget, SG is a free service that doesn't get it's funding from stuff like investors, mainly patreons if I'm correct). If I made this thread, I would've had added to the useful api part the following:
ability to mass blacklist for people below a specific CV (example: blacklisting people below 0.1 Cv, ruling out some extremely common autojoiners) and to be able to create mass amount of giveaways (useful for trains, no need to rely as much on sgtools.info and ESGST, giving away whole bundles). I see the api a bit of a whole for bad actors to exploit though, so that's the main concern for me. Native dark mode on the website should be the no. 1 priority though :)

2 years ago
Permalink

Comment has been collapsed.

it's something we probably wouldn't see implemented soon

that's why I ended with:

Even though I have very little hope that we will ever get an API, I still had to create this thread, I guess to blow off some steam. Actually, I know we are not going to get an API anytime soon.

plus the server load will probably be higher plus costs

Quite the opposite.
See the points above. I obviously haven't done any testing on the SG server, as that's not possible for me, but web scrapping puts more load on the server than APIs.
Scrapping, even when done efficiently, will require the server to at least fetch the data and then dynamically generate the static HTML page, with all the extra stuff, that the scrapper will not require. API on the other hand will just fetch the data and server it as JSON or something.

ability to mass blacklist for people below a specific CV
and to be able to create mass amount of giveaways

That would mean that the API can be used to DO things (i.e. change data), not just retrieve existing information. That brings in a lot more complexity and security concerns. That's is why the first point in the "What should the API NOT offer" section is

The API data should be read-only. That means it should NOT be possible to enter GAs, create GAs, make comments, or make any kind of changes.

.

Native dark mode on the website should be the no. 1 priority though

I am sure that even that will never get implemented. Or a mobile site.
Well, there's always Dark Reader for dark mode. It doesn't work perfectly for SG, but if you know CSS, you can tweak the little things that it messes up.

2 years ago
Permalink

Comment has been collapsed.

Yeah, you do have some good points I haven't considered fully. I don't consider myself a web developer or a server maintenance person, I wasen't fully sure if an API causes more stress than a web scraper, it probably depends a lot on functions and the way in which is implemented, in the end I'm the kind of guy to mess up making a Minecraft server.
My ideas depend a lot on what CJ wants the api to do. If he'll make it and use some ideas from here, it'd probably be made with his own spin on it.
For dark mode on the website I use a Tempermonkey script, as I'm too lazy to go and instll the Stylus extension to make it function the best (the website is usually 50/50 dark mode, it loads very slowly so I should probably get the Stylus version).

2 years ago
Permalink

Comment has been collapsed.

I wasen't fully sure if an API causes more stress than a web scraper, it probably depends a lot on functions and the way in which is implemented,

I can't say for certain too, since as you said it depends on how things are implemented. But unless you really mess things up, ideally and API should use less resources that scrapping.

My ideas depend a lot on what CJ wants the api to do. If he'll make it and use some ideas from here, it'd probably be made with his own spin on it.

From what I have noticed and heard, cg doesn't work a lot on this site. So I doubt it will ever get implemented.

For dark mode on the website I use a Tempermonkey script, as I'm too lazy to go and instll the Stylus extension to make it function the best (the website is usually 50/50 dark mode, it loads very slowly so I should probably get the Stylus version).

Dark Reader? Did you check it out? It's almost a 1-click solution.

2 years ago
Permalink

Comment has been collapsed.

I don't have much to say for the first two, didn't try dark reader as most of the websites I use have native dark mode so no need besides SG.

2 years ago
Permalink

Comment has been collapsed.

I see. Np.
Well if you ever happen to need dark themes on multiple sites, give it a try (I even use it on selected Google Docs and Google Sheets).

PS: You can use whitelist mode to run it only on whitelisted websites/pages.

2 years ago
Permalink

Comment has been collapsed.

Deleted

This comment was deleted 1 year ago.

2 years ago
Permalink

Comment has been collapsed.

Have been away from Steamgifts for a while, a bit surprised we have this now.
Thanks for the info.

2 years ago
Permalink

Comment has been collapsed.

Oh, why is this in discussion and there no proper page for this. Even searching for API on google and SG discussions didn't return this link.
This is very close to the API I wanted SG to support. Thanks a lot for sharing this.

2 years ago
Permalink

Comment has been collapsed.

Just checked, it's not really that helpful, at least to my use case. There are only a few things you can get JSON for. Basically, just list of GAs of a few different types, and list of bundled games.

SGTools already implented them

SGTools checks entries for in a GA, and as far as I could tell, this "JSON support" feature doesn't support getting entries.

2 years ago
Permalink

Comment has been collapsed.

Deleted

This comment was deleted 1 year ago.

2 years ago
Permalink

Comment has been collapsed.

The official thread for SGTools: https://www.steamgifts.com/discussion/b736C/tool-sgtools-new-section-deals
I didn't mean to say that SGTools doesn't use the JSON feature. What I meant is that it can't achieve all it's functiosn from just the JSON feature and some of the data SGTools uses is not supported, hence it imo definitely does scrap at least some of the things, like entries. SGTools checks a lot of things, and it's possible they retrieve them from different source in different ways.

implemented for the most common uses and for more deep purposes, like yours, weren't intended to be made

Not sure what you mean by that?

2 years ago
Permalink

Comment has been collapsed.

Deleted

This comment was deleted 1 year ago.

2 years ago
Permalink

Comment has been collapsed.

Oh, I get it now.
I have already made the tools I wanted to somewhat detect cheaters.
I don't really need an API now. I created this thread because I feel it's stupid that this site doesn't offer an API despite it being well known that there are many tools/scripts that try automating things on the site, enhancing it, or programmatically get data from the site for different reasons.

I wouldn't expect people to make an API for that specific purpose

I doubt there can me a API specifically for determining cheaters. Well, there can be but what I mean is that it will serve no purpose because if you can make an API specific for this purpose, you already know how to detect cheater, so why not just ban them yourself.

2 years ago
Permalink

Comment has been collapsed.

Sign in through Steam to add a comment.