Scala Scripting

In this blog post, I describe how I use Scala for mundane scripting instead of monadic computations.

This semester, I am teaching a very bright and motivated group of students at the Ho Chi Minh Technology of University. (It's the standard undergraduate programming langugages course, and I use Scala as the primary language. But that's for another blog.) While I have every reason to believe that the students are not only bright but also honest, I figured it's best not to leave anything to chance for the midterm exam. Their classroom is very crowded, after all, and I don't want anyone to gain useful information from an accidental glance at their neighbor's monitor. I decided to make lots of slightly different versions of the same exam. But I am lazy too, so here is how I used a bit of Scala scripting to turn two versions of the exam into 64 different ones.

I wrote the exam, then made a copy and introduced small changes in each of the six questions—changes that would require you to look closely at the question text, such as changing names or the order of parameters.

At first, I was going to cut each file into pieces and concatenate them with some shell commands, but then I worried—what if there was a bug, and I had to repeat the operation? I had to automate. I used Scala, mostly because that's what I always do these days, so I'll get better at it.

It's easy enough to read all lines of a file into an array:

val lines = scala.io.Source.fromFile("exam1.html").getLines.toArray

It's also easy to find out where to make the first cut.

val n = lines.indexWhere(_.trim.startsWith("<li>")

But then what? I don't want to iterate. To iterate is human; to recurse, divine. But in Scala, divine isn't best. It's even better to use some built-in function. That's where I got stuck for a little while. There was no built-in function for giving all matching index values.

So, I looked for a function that would give me an array of arrays, and the closest I found was

def groupBy[K](f: (A) ⇒ K): Map[K, Array[A]]

Partitions this array into a map of arrays according to some discriminator function.

Could I come up with some function that gives me different numbers for lines in different segments? Sure, that's easy if not particularly functional:

def pred(l) = l.trim.startsWith("<li>")
var count = 0
lines.map(l => { if (pred(l)) count += 1; count })

This yields Array(0,0,0,...0,1,1,1,....1,2,2,2,....), with the jumps at the lines that match the predicate.

Then the segments of the first exam are

var count = 0;
val segs1 = lines.groupBy(l => { if (pred(l)) count += 1; count })

Truth be told, it took me a few minutes of fussing in the Scala interpreter (affectionately called the REPL). The key to using the REPL effectively is to keep a lab notebook, just like in your college chemistry lab. I run the REPL in one window and keep notes in another. Whenever something works, I copy from the REPL to the notes and tidy it up, by writing a function.

def getSegments(file : String, pred : (String) => Boolean) = {
  val lines = scala.io.Source.fromFile(file).getLines.toArray
  var count = 0; 
  lines.groupBy(l => { if (pred(l)) count += 1; count });
}

Then I copy it back to the REPL and move on.

Now I have the segments from both exams. Next, for each subset of index values, I'll write a different file.

How do I get all subsets of a set? I couldn't find a function for that, so I'll have to be divine for a moment:

def subsets[T](s : Seq[T]) : List[List[T]] = 
  if (s.size == 0) List(List()) else { 
    val tailSubsets = subsets(s.tail); 
    tailSubsets ++ tailSubsets.map(s.head :: _) 
  }

For each subset s of 1 until segs1.size, I want to write an exam. The header is always the same—the lines before the first split point:

val out = new java.io.PrintWriter(outname)
segs1(0).foreach(out.println(_))

Then I write the other sections, picking either segs1 or segs2, depending on whether the index is in my subset:

for (i <- 1 until segs1.size) {
  val segs = if (s.contains(i)) segs1 else segs2
  segs(i).foreach(out.println(_))
}

That's it. After some tidying up, I have a 30 line script that I can run again when my assistant spots the inevitable typos in the exam.

Could I have done it in Java? Sure, but not in 30 lines, and not in 45 minutes. What's more interesting is what it tells us about Scala. I keep hearing that only an academic type theorist can use Scala, but I don't buy it. A blue-collar programmer can do what I just did, with a few tips.

Bulk operations are your friend. Try working with entire collections instead of iterating over the elements.
The bread and butter of working with collections in bulk is map. I experimented with lines.map(...) and a few functions before I hit upon the function that I used for splitting. (When the map function argument yields no result, use foreach instead.)
For simple use cases, such as predicates and map functions, functional programming is really easy—easier than inner classes in Java. Just pass the code that you want to be applied to each element:
```
lines.indexWhere(_.trim.startsWith("<li>")
```
Use the REPL to experiment. I find it enjoyable to build up my task from small snippets. It feels good whenever one of them works, and I am motivated to keep going.
Don't worry about the exotic parts. Gosling doesn't understand them either :-) For example, why can you write lines.indexWhere(_.trim.startsWith("<li>")), but you get an error with if (_.trim.startsWith("<li>")) ...? I have a vague idea, but I am not going to the spec right now to find out.

Why would the blue-collar programmer bother?

Shorter code is easier to understand and maintain. Compare

val n = lines.indexWhere(_.trim.startsWith("<li>")

and

int n = -1;
for (int i = 0; n == -1 && i < lines.length; i++)
  if (lines[i].trim.startsWith("<li>")) n = i;

It's easier to get good at one language than at many. Scala, as the name suggests, aims to be scalable from the most humble tasks (such as my exam generator) all the way to massive frameworks.