Friday, February 08, 2008

Linguistic Success

There's a new favorite pastime on the fringes of the Scala community. That pastime is blogging about aspects of the Scala language that will prevent it from being a "success" unless they are quickly addressed. "Success" is roughly defined as "widespread commercial use." A good metric might be: "How hard is it to find a job programming Scala?" The premise of the criticism usually revolves around one or more complicated, terse, or "foreign" (relative to the author) constructs that are common Scala, or at least favorites among frequent posters, and how these constructs will prevent "average" programmers from understanding Scala and thereby prevent commercial adoption. A recent favorite is symbolic function names (/: and :\ for folds).

The logic of this argument seems relatively sound. When choosing a technology for a project, it is very important to consider the availability of potential employees who know that technology. Forcing every new member to scale a potentially steep learning curve is a frightening prospect. I can imagine development managers lying awake at night fearing that their expert in some obscure technology will leave, and it will take months to replace him. It's a legitimate, although I think slightly exaggerated, fear.

That being said, I think it has little to do with the adoption of a new language. The majority of programmers simply do not spend their free time learning new languages. Many, maybe most, won't even use their free time to learn languages that they know have ready pre-existing demand. They learn when they are paid to learn, or when failing to learn will lead to immediate negative consequences for their career. I think the same can be said of most professions. Most people work because they have to, not because they want to.

Consequently, I think expanding beyond a core of enthusiasts is very difficult, if not impossible, simply be attracting people to the language. Right now some leading-edge Java people are taking a look at Scala, because they think it might be the next big thing in Java-land. These people are different than enthusiasts. Enthusiasts will learn a language for the sake of learning it. The leading-edge folks learn it as a high risk investment. If they can get a head start on the next big thing, it will be great for their careers (and businesses). These people constantly think "can I sell using this technology?" and "if I do sell it, while it come back to haunt me?" This is a very pragmatic perspective, and it is the perspective I take when I'm at work.

Confusing, odd-ball language features make the sell a lot harder. Pitching them as features increases personal risk.

But it doesn't matter.

Why? Because the vast majority of developers are not going to learn a new language because they want to, they are going to learn it because they have to. Not to mention that there are countless languages out there, so going after the enthusiasts and leading-edgers (who are mostly lookers anyway) is just fishing in an already over-fished pond.

So how does a language become a success?

Enough people use it for real projects. Let's say a consultant rapidly prototypes an application using the technology, and that application makes it into production. Now maintenance programmers have to learn that technology. Sure, it's complicated, but unlike the guy learning it on free weekends, the maintenance programmers have all-day every-day. It's hard to learn complex concepts a hour or two at a time, but these guys have all day, and the next day. It's hard to memorize something by looking at it a couple hours a week, but spend all day staring and it and it will click. Not to mention that their livelihoods depend on it. Any sort of "cowboy" development team can cause this to happen, and frankly such teams are pretty common.

So maybe one-in-five maintenance programmers actually like the technology, and admire the cowboys, so when they go get a chance to do new development, they use it, too.

The same thing can happen with products from startups. Let's say a startup builds a piece of enterprise software using Scala. They sell it to big, conservative companies by emphasizing the Java aspect. They sell customization services, too. And then it's back to the maintenance programmer, who has no choice.

Notice a pattern? The key to language success is making it powerful enough for a couple cowboys to do the work of an entire team in a shorter period of time. Selling fast and cheap is easy. If you have enough fast and cheap, the business people won't care if you are making it out of bubble-gum and duct-tape, because you are giving them what they want.

The key to success is making the reward justify the risk. Judging by what some people using Scala for real-world projects are saying, and my one hands-on experience, I think Scala offers it. It's just a matter of time before it sneaks its way into enterprises, just like Ruby has.

Sphere: Related Content


Josh Berry said...

I'm a relatively new lurker to the Scala community, so I don't know if I even have a full 2 cents. So.. yeah, please keep in mind I'm not trying to attack anything here.

If you take such a cynical view of successful languages, wouldn't the real key to success come in corporate sponsorship? The only reason Java is as successful as it is, is because they got a TON of academic and corporate penetration thanks to Sun's muscle (back in the day). This lead directly to a lot of people getting things done using the language, which sorta fed back on itself.

More directly, are there good examples of "successful" languages that have succeeded in a grass roots fashion?

p.s. Oddly, the best thing I have learned from the whole recent bout on /: is just how useful folds can be. :)

Erik Engbrecht said...

I didn't intend the view the be cynical, although it certainly could be taken that way.

But yes, the real key to commercial success is corporate sponsorship, although I think with an open source project that's kind of a chicken-and-the-egg problem. You need a certain amount of demonstrated success before they will sponsor you.

I would say Python and Ruby have succeeded in a grass-roots fashion, even though neither has anything close to the job-share of Java, C#, C++, etc. I'm not familiar enough with the history of Perl, but it probably did as well.

I don't think Scala can go the route of Java and C#. It's too complicated. If you already know some OO programming, you can learned Java in a day. Probably less. It makes it easier to sell. Scala is more like C++, and for many people C++ is still just C with classes, even though it has grown to be so much more. Scala is to Java as C++ is to C.

Josh Berry said...

Hmm... I don't know if I would consider either Python or Ruby to be that large of a success. Though, I'll be honest and say I have no numbers to back this up. (And also, I'm not saying I dislike either, just that neither are a success in the way that Java/C#/C++ are.)

And my real point was that Java became successful because of a huge push in colleges to use it combined with a push (at the same time) in corporations to adopt it. This double push not only created a decent job pool, but a decent worker pool at the same time.

This can be seen with Scala, in no small part. The places where it has amazing popularity right now seem to be among the places where it is taught. So, we sort of have the worker pool building. Now we just need to create the work. (Exactly the "chicken and egg" that you refer to.) I have hopes for Lift, but web frameworks tend to have more borrowed from them then they do actual work done with them. (Unless MS graces them.)

BTW... I guess I should point out that I am quite the cynic when it comes to this. I acknowledge that corporations can bring a lot to the table, but it annoys me how so many industry programmers think they invented everything, as well.

Erik Engbrecht said...

Defining success is difficult. I frequently use Python at work, I work for a huge company, and I'm not worried about defending myself over the choice or getting others up to speed on it. There are books on it and tons of websites with really good examples. You can learn Python without buying a book. There are small companies built around it, and many large ones use it.

Scala has a long way to go in order to get there. I'm actually a little afraid it is going to fast. There is no book yet, web examples are sparse and often too trivially or overly complex, tools are way behind what people expect, binary compatibility breaks between point releases, etc. Yet it's generating a ton of buzz. Don't get me wrong, I love the language, but I think it could use some time to stabilize.

Erik Engbrecht said...

Note: I was defining why I think Python is successful in the first paragraph of my last comment.

Josh Berry said...

I can definitely understand the difficulty in defining success. I agree with you whole heartedly on this. (I think this is actually one of the main reasons I love the interest in languages like Scala. Most people that love it, do so because they truly believe it is the better tool.)

I do have some worry about you claiming huge success with Python based on anecdote. I used awk and sed extensively at one job, but I wouldn't go so far as to say they are a huge success.

And I must say that I think I agree with you 100%. Please don't let my musings on the comments mislead you on this. I am incredibly excited about Scala for -- I believe -- all of the same reasons you are. My only regret is that I do not see a good place to contribute, yet. :(

Erik Engbrecht said...

I wouldn't say Python is a huge success. I would say it is a success. Comparing Python to Java is kind of like comparing someone like Eric Sink or Joel Spolsky to Bill Gates.

On a side note, it wouldn't surprise me if Sed and Awk have greater usage than Python. They are everywhere in Unix shell scripts. It's probably been a very long time since anyone made a dime off of them (besides writing code and books), but if they disappeared tomorrow a bunch of our infrastructure would need serious patching.

Thanks for the comments.

Anonymous said...

josh berry - "My only regret is that I do not see a good place to contribute, yet. :("

Do you know anything about writing Eclipse plugins? The existing one is decent at what it does, but it is limited compared to what it should/could do. It could use some work, and I hope to help it along. I hope others, like you, will join me because of my next comment.

As far as Scala's success is concerned, I work for an enterprise technology-based company which has been investigating its use. The decision is to hold off for now, and to keep an eye on it. I spoke with our Lead Architect about Scala, and he said it is "IDE support and one useful framework" away from widespread adoption. This is saying a lot, but I mentioned to him that a framework may not even be necessary because one could use existing Java frameworks. For example, I wrote a little Seam application in Scala yesterday as a proof of concept.

Josh Berry said...

Honestly, not much. I have been highly interested in doing this in the past, just never had a large item to make a plugin for. So it is something I am highly interested in learning. Do you have a link?

Erik Engbrecht said...

The Eclipse plugin definitely could use help, but thus far Sean McDirmid, its creator, has been reluctant to set it free until it reaches a more stable state, and it is taking its time in reaching that state. But it is available in svn.

I would suggest working on some API docs, which is an ongoing problem. See the following thread:

Anonymous said...

I too would bet that you cannot take most languages critical mass unless they have some sort of profile within 'x' number of multinationals. (Non-compiled stuff like PHP might argue otherwise, but I personally think it is a bit of a push to say that PHP driven websites == software.)

I think that thinking along the lines of taking things critical is quite interesting with regards to Scala.

When launching any new business, you have to have at least one tangible and meaningful differentiator. If you actually want to make some serious money, the conventional approach is to become as generic as possible. For example, as a founder, you definitely want to work yourself out of a job before trying to flip the business. The reason for this that you can sell or float the business without being paying too high a penalty in relation to integration risk etc.

So, you might lead by having better, and different, technology (and, by extension, better everything else), but what will make you money is the more traditional assets - customer list (which cannot rely on one mega salesperson etc.), fixed assets, 'brand' etc.

The only time you can realistically be more ambitious than this is if you can do something that is extremely hard to replicate. For today's averagely smart person, coding is not difficult; Scala might be just about difficult and useful enough that you could make a play for the big time.

So, that possibility might make a difference. But, regardless of individual cases, it seems difficult to imagine maintenance coders overseeing Scala codebases anytime soon.



uncial said...

The other path to fame and glory for an humble programming language seems to be a highly visible "killer app". I don't think that's sufficient,BTW, but it might help kick-start some momentum.

CGI did it for Perl; Rails seems to have done it for Ruby.

Is there a sufficiently-interesting problem domain for which Scala's key strengths could provide similar opportunity?

(And I certainly don't want to slight David Pollak's work on lift, but wonder if too many would simply say "Oh. Another Rails." and miss the point.)