Operator Overloading Considered Challenging

In this article, I explain why I think that operator overloading is a good feature, even though it is hard to get right. To illustrate my point, I go into more detail of the operators in the Scala collection library than you want to know.

Java has no operator overloading. I always thought that was a shame. For example, BigDecimal would be a lot more popular if you could write a * b instead of a.multiply(b).

Why doesn't Java have operator overloading? Well,  C++ has it, and people kept saying that it makes code hard to read. Actually, in C++, you can have a library class vector and write v[i], thanks to operator overloading. In Java, we have unsightly v.get(i) and v.set(i, newValue). Easier to read? I think not.

Of course, someone somewhere out there abused operator overloading in C++, doing something silly like overloading % for computing percentages. The horror. It's actually pretty hard to do much abuse in C++ because you can only overload the standard operators. Personally, I never ran into anything scary

In Scala, on the other hand, you can define any operators you like. If you want to check for five star hotels, you can define a predicate *****. Unicode is fair game too: fred ♥ wilma.

Scala detractors foam at the mouth when they see

(1 /: (2 until 10)) (_ * _) 

Actually, that's unfair. Let's write it without operators:

2.until(10).foldLeft(1)((x, y) => x * y)

It still looks like magic if you don't know foldLeft. And if you do, the operator version is simpler.

(BTW, this computes 1 * 2 * 3 * 4 * ... * 10.)

So, I firmly believed that operators are a good thing when used with restraint.

I still believe that, but I came to realize that restraint is harder than I thought.

Some fellow had kvetched how the Scala collections library has too many crazy operators. And I remembered some discomfort when I wrote the chapter on collections for “Scala for the Impatient”. I put together a table with all operators for adding or removing elements, and it did seem a bit of a mess. I pointed this out on the Scala mailing list, and Martin Odersky asked what I suggested to fix it.

Well, fixing inconsistencies happens to be my forte, so I went right at it.

For starters, we have the following:

coll :+ elem // Makes a new collection, appending elem after coll
elem +: coll // The same, but elem gets prepended

It's a nifty trick in Scala that an operator ending in a colon is right-associative, and elem +: coll is really the same as invoking the +: method on coll.

You can also insert elements in bulk:

coll ++ coll2
coll2 ++: coll

Did you notice the asymmetry? Why isn't it :++ for the first one? I pointed that out and was told that ++ is prettier and shorter. 

What if the collection is a set? Then there is no intrinsic ordering, so it seems silly to distinguish between appending and prepending. You just write

set + elem

Except, of course, when the set happens to contain strings.

Set("Fred") + "Wilma"

doesn't add "Wilma" to the set, but it coerces Set("Fred") to a string and concatenates the two. Ouch.

So far, these operators return a new collection, leaving the original unchanged. That's the functional way, and it's often good. But sometimes you want to mutate a collection. For example,

buffer += elem
buffer ++= coll

Did you notice the inconsistency? To append without mutating, it's buffer :+ elem, so for consistency's sake, it should be

buffer :+= elem

I asked why it wasn't so and was told that += is prettier and shorter. Yes, it is, but actually there is a :+=, because Scala always synthesizes an op= from any operator. And it's subtly different from +=.

What about prepend-and-mutate? The operators must have a colon at the back, so they are +=: and ++=:. Why not =+: and =++:? Gentle reader, if at this point you lost the will to live, I hardly blame you. 

Just one more thing before I come to my conclusion.

For lists, the prepend operator is ::, because, well, it's always been so. For example, 1 :: 2 :: 3 :: Nil makes List(1, 2, 3). And we don't want to change it to +: because, well, it's never been that. And we don't want to replace +: with :: because then we lose the beautiful symmetry with :+, even though we don't care about that symmetry when it comes to :++ or :+=.

No, I couldn't come up with a fix that was consistent, pretty, and compatible with the past. But I learned something from the process.

  1. Don't mess with +. String concatenation has ruined it for the rest of us.
  2. When your operator starts looking like Morse code, give up. You don't have to have an operator for everything.
  3. Use asymmetric operators for asymmetric operations. :: for cons, or | for shell pipes, are bad role models.
  4. You have one chance. Operators are powerful stuff. Once people are used to :: or | or !=, they will refuse to switch.

Is Scala wrong to have operator overloading? No, on the contrary. Operators are incredibly useful. They are just really difficult to get right.

We know this from mathematics, where of course operators abound because they are so useful.  New operators get created all the time, and many of them sink into the obscurity that they richly deserve (such as Newton's fluxion notation). Nevertheless, some awful operators survive. Consider derivatives. Input: A function. Output: Another function.  We have two operators: f' and df/dx. The first is inadequate and the second is cumbersome. As an undergraduate, I had a textbook that bravely soldiered on with Dxf, which made a lot more sense, but it was too far from the mainstream.

In summary, operator overloading is neither a mistake nor a panacea. I want it and I want people to use it wisely. As I learned, that's harder than it appears.