14 January 2013

API Scraper written in Scala

I redid the Ruby script I made, in Scala.  This is the scala version... it seems to work better with less duplicate data:

import scala.util.parsing.json._
object Parser {
  def main(args:Array[String]){
    for (i <- 1 until 1000) {
      try {
        val json = JSON.parseFull(scala.io.Source.fromURL("http://us.battle.net/api/wow/quest/"+i).getLines().mkString("\n"))
        json foreach {println}
      } catch {
        case e: Exception =>
      }
    }
  }
}


It works like this.... it's pretty simple.  I'm using the scala.util.parsing.json library to parse some json i'll be getting back from a public API.

I start with a for loop and say i = 1, lets increment this till we reach a value (I picked 1000.)
Within the loop I do a try, because the try will let me catch exceptions... I already know that not all values passed into the api will work. some id's will be a 404.  I dont want the app to stop because it fails on a id.

I set a immutable variable (val) to the JSON.parseFull.... which ultimately goes to WoW's quest API and I append a: +i.  The i, being the value here that's incrementing in the for loop.

For each result, I print it... using foreach.

Since I'm using "try" i'm also using "catch" and making sure I catch exceptions.... and move on.

No comments:

Post a Comment