Python/Ruby: script languages, nothing more.
rzwitserloot posted in programming on October 1st, 2006
Tim Bray of sun has been interviewing the JRuby guys and stumbled upon the crux of why Python and Ruby are effectively sucktastic when it comes to implementing projects of more than, say, 6 manmonths. They don’t play ball with the IDE. Rather unfortunately, this conclusion leads him to the wrong realization. See here, the fantastic delusions of an evangelist.
To summarize, he advocates letting the IDE watch over the shoulder of the interpreter as it runs through a unit test suite. That way, the IDE can identify for each variable it finds in the code which types of values it ever holds. (Ruby/Python aren’t weakly typed; each object does have a very specific type. It’s just references to them are untyped, so for any given occurence of a reference in the code, the IDE has no clue what it could contain. Objects exist only at runtime, not at compile (read: writing code) time).
Hold the phone! Are you nuts? Short of 100% code coverage any refactorings are always an unknown, and any bugs caused by it will by definition not be caught by any of your unit tests. Reaching 100% code coverage itself is waaaaaaaaaaaaaaaaaaaaaaaaaaaaay (I cannot stress enough by how many factors!) more work compared to the very mild job of adding types to all your declarations, something your IDE practically does for you, I might add. Not to mention the time it saves you on debugging.
I’m beginning to believe that python/ruby folk are so obsessed with unit tests exactly because they can’t trust their compiler any farther than they can throw it, because it can’t help but see your code as a pile of text instead of a structured set of links. (For you agile fanboys out there who believe my statements that unit tests aren’t all that useful and generally a waste of time, remember, agile blows).
Tim, and all others convinced by the droning of the evangelists out there: It’s really quite simple. An IDE is a fantastically useful device to work with, and purposefully eliminating any chance your IDE has of becoming a second pair of eyes (because, let’s face it, nobody really does pair programming, it just sounds nice) is a monumental error.
Python and Ruby are scripting languages because they do not scale along the development process. Working on a big project with more than one person very quickly spirals out of control in languages like that, without an IDE to keep ordnung. I’m not saying java is freed of all blame here - there’s loads of boilerplate and syntax issues that can be cleared up (without ruining the strengths of java like Gafter’s and Gosling’s latest brainfart, The closures for java proposal), and there’s a general feeling amongst the community that convoluted forests of factories and XML settings files make for ‘flexible’ code. Unfortunate. At least these issues are fixable.
Weaning python and ruby off of their obsession with dynamics is impossible.
October 1st, 2006 at 12:56
Wow, quite a flamin’ entry you got here. I’ll take your bait since I’ve nothing better to do at the moment:
“nobody really does pair programming” - naturally, lots of people do it. But not necessarily 100% of the time. Pair programming is, IMHO, a great way of tackling a unusually hard problem, for example. And getting to know an existing system is less painful if you pair with someone who has experience with the system first.
The whole “Agile blows” -thing is getting really old, really. But it’s mostly chanted by people who think Agile equals XP. In essence, Agile is about being pragmatic and not doing any unnecessary work. It requires an open mind, lots of self-intiative and is definitely not for people who have to be constantly herded. But you knew that already.
Ruby is a really nice language. I’m with you on one thing: at the moment, I wouldn’t feel totally comfortable building any huge applications with Ruby, exactly for the aforementioned reasons of poor IDE tools etc.
But, to think that the brilliant minds who’ve created the bustling ecosystem that is the Ruby community won’t be coming up with better tools for Ruby development in the future is kind of unrealistic, don’t you think? Smalltalk had the refactoring browser, so something similar should be doable in Ruby.
October 1st, 2006 at 13:30
I think you’re missing the point that Bray and others are actually admitting that IDEs are useful and are thinking of ways to extend that usefulness to dynamic languages.
Saving records of run-time information and making that visible seems like an actual good idea to me.
Rather than bashing dynamic languages every chance you get, I think your views might get more adoption if you cleaned your own house. Java is wholly too cumbersome as it is and if you say that you yourself would abandon the language if the closure proposal gets in, I don’t know why we should board a sinking ship.
Take a hint from C# (especially 3.0) which incorporates lots of features which make programmers’ lifes easier without throwing out your precious type safety.
October 1st, 2006 at 14:16
Writing tests isn’t s much work, and it’s useful work too. Typing variables doesn’t give usage examples of your code, nor does it document behavioral assumptions. In fact, most of the time, types just restrict the polymorphism and clutter the code.
Also I don’t remember saving debugging time because of types… in fact the first time I really used the debugger (instead of applying the good old printf method
was in Smalltalk, and IMHO because it’s way better integrated and flexible than in Eclipse… static types aren’t really part of this probem.
October 1st, 2006 at 20:59
Re agile blows: All true, and guilty as charged. However, the notion that you have to write unit tests for every last little nook and cranny of your code is generally right up there with mindless adherence to XP or other fads that don’t actually work in real life. My comment was intented as a harsh putdown to that crowd.
pair programming: As a way to learn, it’s brilliant. Practically though, the vast majority of code written is done on your own. Except at least your IDE is reading along and helping you out, which helps. A lot.
My point, though: As bray already realized - you CANT write eclipse for ruby. The ’solution’ of forcing you to write 100% coverage unit tests is unrealistic. It’s a serious problem and there ARE solutions to this without completely breaking the mindset behind ruby and python - though I doubt you can adapt either language at this point of time.
boo has its own problems but it reads and acts a lot like java, BUT its strategy of agressive type inference instead of completely typeless variables does mean an IDE could be written for it that knows almost as much as you do about your code. (http://boo.codehaus.org/). I can see a language like that going toe to toe with java for the large project market. Not Python/Ruby.
October 1st, 2006 at 21:04
Re Alper:
C# sucks. Read this actual C# feature for picking the first non-null item in a list (http://blogs.x2line.com/al/archive/2005/07/22/1178.aspx) and I’m sure you’ll start thinking about C# the same way I do: on the fast track to PERL-SOUP land. Adding an operator or language feature for every whim or fad that comes along makes your language evolve itself right out of utility very quickly.
I don’t bash Python or Ruby for what they are. I bash the delusions by the majority of P/R evangelists that seem to think they beat java in every way. Far from it - P/R has its bailiwick where they work well, but taking on the enterprise isn’t in the cards.
C# uses the monkey house method to adding features:
They throw a lot of shit against the wall and wait for some of it to stick.
A tiny amount of those many changes look useful, but without long and hard deliberation before adding to a language, you’re dooming it to obscurity. I like all of java5’s features, and every last one of them was thought through extensively - they embody changes to java people have been clamoring for (and faking by way of libraries) for half a decade or more. That kind of deliberation period, to see if an improvement is truly needed, is a good thing. It makes your language a bit slow on the uptake of the hip new thing, perhaps, but you’ll just have to deal with IDE templates until then.
October 1st, 2006 at 21:13
Re Damien: Writing tests for 100% coverage is easily 10x more work than writing the code yourself. Have you actually tried taking an existing large project (or starting a new large one) and writing full, 100% coverage tests for the whole shebang? Come back when you have.
Typing gives lots of useful hints about implementation. If I see a getUsers() method, and I see that it returns a List<User> instead of a List<String> or even in python/ruby land, ’something or nothing, who knows?’, I already know most, if not all, of what I’m to use that method for. I can always ctrl+click on ‘User’ and immediatly warp to the User class, check out what’s in there.
Pre-generics, you never knew what you got without looking at the javadoc or inspecting the objects in your list. Generics is a Good Thing - it adds documentation of a kind that the compiler ensures remains in sync with your source.
Here’s an exercise for you: Write a pydoc for a python function according to how Guido and crew think you ought to write one: include the ‘return type’ in the docs.
Now write a unit test for it too. A month later, change the return type from say String to a User object, because your user logistics got a bit more complicated. Remember that just like in Java, objects can provide a string representation, and doubtless your User object does just that. So, your unit test never tells you there’s a problem, BUT, there IS a problem. Your pydoc is out of date.
In java that stuff never happends, because things that you pydoc in python, you specify in the language itself in java. Hence, the function of generics is not so much to prevent class cast exceptions (which, as so many have said, doesn’t happen all that often), it’s a device used to both document what a structure contains, an insurance that this ‘documentation’ never becomes outdated, and an officialized notation that allows all IDEs to parse this data and help you along as you type.
Polymorphism is not at odds with static typing. Look up ‘refactor script’. Any simple changes that are prevented due to the static typing getting in the way are all automatically refactored into the new thing - AND you get nice immediate notification of where the refactor doesn’t work. Exactly the places where a dynamically-typed language would silently continue and fail only at runtime.
As far as your debugger comments: typing is mostly useful while you are writing code, not debugging it.
October 1st, 2006 at 23:19
@Reinier:
C# is cool: I don’t think the ?? operator is that strange or hideous. It is perfectly in line with the nullable types feature which is a way to make databases mesh easier with the language (as is LINQ). I’m not a big fan of the .NET framework but they are doing a whole lot of things right with C# and the CLR.
You would rather have the complete lack of will displayed by Java to extend operator overloading even for the most obvious of cases?
100% coverage: And about the 100% coverage thing. Nowhere does Tim Bray say that you have to have 100% coverage. Of course you need it for full refactoring but getting the partial benefits on the way may be nice or enough already.
Python example: Your example doesn’t jive. Comparing a string to an object with a string representation (__str__) returns False unless you first pass your object through the str() function.
Java allows a lot of sloppiness in its automatic type coercions. Python for one does not.
October 2nd, 2006 at 0:01
You think the ?? operation is OK? It’s ridiculous! Here’s a 3-line function that does the EXACT SAME THING: (java)
public static <T> T coalesce(T... items) { if ( items == null ) return null; for ( T item : items ) if ( item != null ) return item; return null; }Let’s see now, the advantages of the operator versus this thing. All I can think of is that it saves you a few characters. In return you get the ‘??’ operator. Go out, ask any programmer what the ‘.’ operator might do. I bet a load of people will figure out it probably unrolls some sort of reference. Ask the same about any mathemetical operator. You can even go for the archaic ‘&’, which is known widely if only because C defined it and so many languages joined in; even bash shell scripts.
Now ask what ‘??’ might do. I bet virtually no one will be able to tell you. If I read this in code:
A ?? B ?? C
I have NO CLUE what it might mean, I’d have to look it up. This is the very definition of perl soup, and this is why I absolutely refuse to have anything to do with C#. It sucks. It’s a dumb language designed by total idiots.
October 2nd, 2006 at 0:11
As far as java’s lack of will: I’m actually against dynamic overloading of operators, eventhough it can be squared perfectly with the IDE. Not with much conviction (mostly I don’t care), but where something isn’t neccessary and might introduce a lot of confusion, don’t. The java guys so far seem to understand this.
As far as not supporting the basic operators for BigDecimal and BigInteger: Yes, that’s dumb, and java would be a better language if this was added. You’re trying to force me into choosing one or the other. I pick an imaginary language that’s a lot like java but that’s better. I also observe that while java might get to there with small, incremental improvements, the dynamic basis of python and ruby means they’ll never get there.
As far as 100% coverage goes: As I said, if you don’t have 100% coverage, your refactor script will fail exactly for those circumstances which are not covered by unit tests. That’s a fantastic way to write code! Any code not covered by a unit test is pretty much guaranteed broken. Dumb.
Partial benefits - yes, the idea is solid. Of course it is; I preach the benefits of an IDE for good reason, and the partial benefits make editing python/ruby code slightly more fun. But instead of going through all that trouble, the extremely minor task of getting your IDE to generate your typing for you is superior to this plan, and way less work. Once you start needing this, you need to switch to java.
As far as this one:
“Java allows a lot of sloppiness in its automatic type coercions. Python for one does not”
Uh, no. The only sloppy type conversion java engages in is casting doubles to ints and the like, something I’d rather make explicit. Python and Ruby do the exact same thing.
Heck, in java, “hello” == “hello” is not guaranteed to be true, and someObject.equals(someObject.toString()) is always false, unless someObject is itself a string. I have no idea where you dreamed this up.
A direct comparison will not work, but there are plenty of situations where a list changing its type internally to something similar enough to be compatible (especially in dynamic languages, where, after all, you just need to cover whatever methods are called in the unit tests) makes your pydocs go out of sync.
Besides, anytime you are pydoccing a return type, why NOT just lump it in with the method signature. That’s actually less work!
October 3rd, 2006 at 17:39
The 2nd link is broken, there’s a trailing %22
October 3rd, 2006 at 17:43
Fixed.
October 4th, 2006 at 0:24
If typing was a substitute for tests a successful compile would confirm your program works correctly. You’re quite simply not making a fair comparison. Proper tests take longer because they are not as simple and mindless as just checking types.
October 4th, 2006 at 7:07
“Any code not covered by a unit test is pretty much guaranteed broken. Dumb.”
Yes, indeed. (And that’s not an argument for or against a specific programming language.)
October 4th, 2006 at 11:39
a gentle reader wrote:
—
“Any code not covered by a unit test is pretty much guaranteed broken. Dumb.”
Yes, indeed. (And that’s not an argument for or against a specific programming language.)
—-
Wow. Certainly makes you sound smart. Unfortunately that’s total bull. I write tons of code with no unit tests that work fine. Here’s a simple sample for something I’d never write a unit test for:
public String toString() {
return String.format(”User %s”, userName);
}
yet according to your rule this is broken. Bull. There’s also a bunch of code (amongst which, GUI drawing) which is just far less hassle to test by hand instead of in an automated fashion. Yes, oooh, I lose the ability to ‘re-test’, but with eclipse, if I get absolutely no warnings whatsoever during a refactor job, I don’t actually need a unit test to confirm yet again that my code is semantically unchanged. In java, such a thing is then guaranteed.
Gentle reader, I bet you prefer python and ruby. Your mind has been tainted with an overt need to cover everything with unit tests.
October 4th, 2006 at 13:53
“If typing was a substitute for tests a successful compile would confirm your program works correctly. You’re quite simply not making a fair comparison. Proper tests take longer because they are not as simple and mindless as just checking types.”
I never claimed any such thing. I merely claimed that using (100% coverage) unit tests to achieve the same thing as what typing can do is a ridiculous idea.
October 7th, 2006 at 21:31
Wow, are you trying to dethrone hani?
October 8th, 2006 at 3:18
I’ve done some googling. I think I found whom you are referring to. However without a further link with specifics I can’t really answer that.
October 11th, 2006 at 23:12
“writing full, 100% coverage tests for the whole shebang? Come back when you have.”
Heh, first, I try to test first then code. Second, I don’t need 100% coverage, I just need to exercise the system enough, with a variable definition of enough.
“Typing gives lots of useful hints about implementation.”
Maybe but it also forces too early choices.
“If I see a getUsers() method”
Just the name means that it returns a collection of user objects… it would be named getUsersNames() if it returned strings.
“I can always ctrl+click on ‘User’ and immediatly warp to the User class, check out what’s in there.”
I can always ask for senders of this message and see how the method is used.
“include the ‘return type’ in the docs.”
That would be redundant with the code, and I care more about the role of the object (a subset of it’s protocol maybe).
“outdated, and an officialized notation that allows all IDEs to parse this data and help you along as you type.”
Granted, writing precise completion without static types is more difficult, but it works OK for me.
“As far as your debugger comments: typing is mostly useful while you are writing code, not debugging it.”
Isn’t that the same thing ?
October 12th, 2006 at 14:27
Damien: No, ‘exercise the system enough’ doesn’t work - the only workable definition of ‘enough’ is 100%. Any rare conditions not covered by your unit tests will automatically break if any refactoring is done. Refactor scripts only work well when you -know- nothing gets broken by them.
Typing does not force early choices - that’s what refactoring is for. In practice, most changes of types force you to also change callers, but with java, you at least know, for SURE, were to find those callers. In latent-typed languages you don’t know where to look (short of the 100% code coverage plan).
getUsersNames() -> really? Check some real code. You’ll find tons of ambiguous names. You can NOT stuff the entirety of parameters, return type, and potential fault behaviour into one method name, that’s obvious.
checking your own stack works only during runtime. Ctrl+clicking on any identifier in eclipse works as you write the code. The closest that latent typing languages can get is REPL antics, but that isn’t the same thing.
as to redundancy of putting return type in the docs: The official guideline for pydocs disagree with you there. I’m guessing Guido is better at python than you are.
typing and debugging are by no means the same thing. To wit: I needed some rather odd approach to parsing and writing JSON. In 90 minutes I wrote the whole shebang from scratch, whilst distracted (watching a House episode), and non-automated, post-written tests then showed the whole thing was bug free. With latent typing that wouldn’t have happend, as I correct plenty of redlines as I was typing.
October 13th, 2006 at 11:15
“Any rare conditions not covered by your unit tests will automatically break”
Yeah well, bugs happen. In this case it’s a bug in the tests… And by the way, how do you specify which behavior your refactoring shouldn’t break? Should that really be just the language semantics or something more on the domain level?
“Typing does not force early choices - that’s what refactoring is for.”
No. Refactoring is for fixing things when you realise some (often good at the time) choices now appear to be bad because your understanding of the system or the requirements have evolved. By forcing early choices I meant forcing early decisions like choosing the name for an interface to declare a variable when you haven’t written the code that uses that variable yet (ie., you don’t know yet which messages you will send to objects in this variable).
“In latent-typed languages you don’t know where to look”
well of course, if the method happens to have a popular name and different homonyms are in the current namespace, I have some guessing to do.
Yes I do, it’s in the contextual menu
“I’m guessing Guido is better at python than you are.”
Yes obviously. And Bjarne is way better at C++ than me. I just happen to have different taste for language design.
“typing and debugging are by no means the same thing.”
I meant programming and debugging.
October 13th, 2006 at 16:11
‘a bug in the tests’? You consider less than 100% code coverage a bug? Get real - you can’t get 100% code coverage for a practical project. It just doesn’t happen, heck, you can prove that you can’t get that far using some creative application of the halting problem.
Refactoring shouldn’t break any behaviour, period. That’s the point and meaning of a refactor job - to restructure code without changing what it does in any way.
Refactoring IS for avoiding the ‘early choice’ dilemma. Why use refactoring only to correct bad design mistakes? Tell you what, I’ll call it a ‘modification script’ instead of a ‘refactor script’. It’s exactly the same thing, just a differenet name. Would that help?
Your exact example is something you can fix in 5 seconds, with 100% guarantee that everything is semantically unchanged, with an eclipse/netbeans/IDEA refactor script. You can NOT PULL THAT TRICK in python, and while in many cases you don’t need any names, you still need a name for your classes, or your functions, and changing those in python is a major headache job. You’ve got it the wrong way around - it’s static typing that helps avoid early choices, and dynamic typing that forces you into them.
As far as your context menu goes: In small hobby projects that’ll work… somewhat. In large system projects, it won’t. In that contextual menu you’ll get every function in your whole app, because the text editor has no clue what ‘x’ might mean when you do x.(hit auto complete button here), so it gives you everything. In java, you eliminate 99.99% of all possible options just by typing your identifiers. The options that remain are then guaranteed.
programming and debugging are the same thing, but obviously you don’t do them at the same time. What kind of weird argument is this for in the first place? I mean, you’re mix and matching word definitions here.
October 14th, 2006 at 11:55
“a bug in the tests”
I mean you don’t need complete test coverage. If you include all possible parameter values in your notion of coverage that is equivalent to testing all possible executions of the program, so it’s absurd. Just relax and accept to break the system from time to time or add a test on demand.
“Refactoring shouldn’t break any behaviour, period”
That’s impossible. Perfectly valid refactorings from a static point of view can break apps that use introspection for instance… in JUnit if you move methods around that will probably change the execution order of the tests. In this case it’s OK but it’s still a change in behaviour. So refactoring really depends on which behavior you want to preserve.
“it’s static typing that helps avoid early choices, and dynamic typing that forces you into them”
Sure you can change a type after the fact, but you did have to think about it in the first place, when what’s actually important is the role of the object, not its static type.
“What kind of weird argument is this for in the first place?”
Simply that the distinction between programming and debugging is not clear-cut. Most of the life of an application is in maintenance, and even in the initial development phase, the earlier bugs are caught, the cheaper they are to fix. So in the end I like to see programming as fixing bugs, the first one being “d’oh, my program doesn’t exist”
October 14th, 2006 at 15:55
Covering all possible parameter types/values would certainly help, but once you make that a practical proposition, you are forced to dump some of python’s dynamics by the wayside. It also forces you to think about the type in the first place. Once you’re going through that trouble, you might as well just type 5 letters and prepend ‘String’ to your function definition, don’t you think?
Refactoring not breaking anything: If you don’t use introspection (and you hardly ever need to) a refactor script is guaranteed not to break anything. For most introspection, eclipse is smart enough to figure out something may be going wrong, and sends you on an inspection tour past all dubious spots where eclipse thinks there might be trouble (reflection-using code that mentions a relevant class or method name in a string, for example). Re-ordering tests is not a refactor, AND, furthermore, if changing the order of the tests changes the operation of your testing suite, the full thing was buggy in the first place. Refactor scripts being so good you don’t have to have a unit test to be sure everything still works is a luxury only available to languages with static typing, not latent typed languages.
As far as picking types when you write goes: You don’t have to. You just write ‘void’ as return type, for example, and then type your code. Once you’ve written the return statement, eclipse shows you an error ( red underline) on the return line, as the return type is by definition not ‘void’. You then hit quick-fix (CTRL+1) and pick whatever makes the most sense off of the suggestions for changing the return type. At this point you must already have made the decision of what you will be returning as you just wrote a return statement.
However, none of that is very important. The ‘role’ of an object is its class, or interface, or what not. This is not nearly as obvious in python, but java’s built that way from the ground up. In python you have a class with a ’shoot’ function and if that’s all you are looking for, that’s enough. In java, you need to square that ‘role’ (anything that has a shoot function) away into a superclass or interface.
The importance of doing it that way becomes much more obvious when I decide to accidentally pass an instance of ‘gun’ ot the function that wants anything with a shoot() function, instead of the slew of ‘camera’ instances I’ve been passing so far. Then you shoot yourself in the foot. Namespacing is nice, python doesn’t have it for method calls. You’re not going to call each python function net.productsite.appname.camera.shoot, are you? That would be ridiculous. In java, effectively, everything does have that name, but it’s mostly invisible, and where it isn’t, eclipse ALWAYS generates it for you (automatic import statement generation).
“the earlier bugs are caught, the cheaper they are to fix.” - amen! You’re exactly right. Which is why I like java so much. Over half the bugs are caught as I write them. You can’t catch em any earlier, and they are indeed stupendously cheap to fix. Usually 3 keystrokes. (CTRL+1, zero, once, or twice a tap on the down button, and enter). Programs do spend a lot of their time in maintainance, yes, which is again an argument to go with static typing. The savings of latent typing occur in a phase that is inconsequential in the ‘big picture’, yet without static typing you lose the ability for a whole lot of fun and useful abstractions from automated tools and IDEs to help you maintain it later.
March 19th, 2007 at 17:43
[…] One of the strengths of GWT is that, due to java’s explicit static typing (I’ve talking about this before) GWT can determine, with very fine granularity, which code is actually used, and which code isn’t. […]