Your actual hurdle is that the game names aren't actually in the HTML page itself (check the source, i.e. the document your program retrieved). The page is coded as a template, and JavaScript is used to populates the table of games via some JavaScript DOM-manipulating libraries. So you need to find out how and from where the page gets the list.
Most browsers today will tell a "lie", when you open the page in developer tools, because the HTML source shown is the generated HTML source. That is, it shows you how the DOM looks after the DOM-manipulating JavaScripts have run.
Comment has been collapsed.
Yeah, I actually tested it before and couldn't figure out why it fails to work. When I tried retrieving all the links from the page, it just gives generic ones like hub and feedback, log in and PayPal. Your point actually makes a lot of sense.
Comment has been collapsed.
There you go: http://steam.tools/cards/set_data.json
Use a json parser or regexp the games out of there.
Comment has been collapsed.
That is what I call raw data. How did you find it though?
Comment has been collapsed.
It's actually called JSON ("raw data" is usually used for the binary data and can't always be converted to text). It's really easy to learn it, but you don't have to: java most certainly have a parser to convert it to an object/dictionary/whateverjavauses.
And imo the easiest way to find it is to check the network tab on the developer tools on Firefox or Chrome. Both have a filter for XHR which should have this kind of stuff.
Comment has been collapsed.
Yeah, I actually looked it up but where ever I try, dev tools doesn't show any information that isn't written on the website itself.
Comment has been collapsed.
Due to the way the script is written, it may not be detected. Hence you'd have to look for it manually by checking every linked script file. Naturally you can skip all the ones containing the JavaScript libraries (unless everything is combined into one JavaScript file, which isn't the case here).
You are looking for the glue, and in this case you'll find it in "main.js" included at line 188, and in "main.js" you'll find the request on line 354.
Comment has been collapsed.
Posting this for EshanKia as he doesn't have an account here.
Hey guys, creator of steam.tools here to answer a few questions:
The reason the data doesn't show up if you download the page directly is because
the database is loaded afterwards through Javascript. As someone else pointed,
that JSON link is the database being loaded, so all the information should be there.
Dev Tools should also show it, it does for me: http://i.imgur.com/retamO6.png
There are various JSON parsers for Java. Google's gson is my personal favorite.
I'm not too knowledgable with Java though, I could help you more if you were working in Python.
If you want to tell me what exactly you're trying to do, I can point you towards
the right direction and where you can get the data you need. Use the "Feedback" button on the site to reach me.
Comment has been collapsed.
Yeah, I just realized my mistake.. I had something searched when I used the dev tools. Basically, what I am trying to do is match names of games with cards with a list of games Tremor Games has, mostly for myself. Kinda make a list of all the games they have that have cards. Mostly I do it for myself to practice getting data from websites.
Comment has been collapsed.
332 Comments - Last post 31 minutes ago by wigglenose
106 Comments - Last post 40 minutes ago by Cim
294 Comments - Last post 1 hour ago by GraVe23
223 Comments - Last post 1 hour ago by Kingsajz
34 Comments - Last post 3 hours ago by Formidolosus
16,725 Comments - Last post 4 hours ago by Kenruyoh
49 Comments - Last post 6 hours ago by wigglenose
65 Comments - Last post 4 seconds ago by Boson
74 Comments - Last post 13 minutes ago by s4k1s
1,437 Comments - Last post 17 minutes ago by s4k1s
114 Comments - Last post 19 minutes ago by Tiajma
137 Comments - Last post 20 minutes ago by uchihaitachics
2,827 Comments - Last post 27 minutes ago by eeev
1,605 Comments - Last post 1 hour ago by Vasharal
It is something I find very useful but yet I can't seem to figure out how to do so..
I've been looking at Jsoup all day and having a bit of problem with it. I am trying to get game names from http://steam.tools/cards/ just to play around with it but can't manage to do so.
Anyone has a clue how to do so?
Basically the code starts like this:
String url = "http://steam.tools/cards/";
Document doc = Jsoup.connect(url).get();
Next should be getting elements but that part seems to be what fails for me.
Elements items = doc.select("span[class=game]");
Comment has been collapsed.