Here’s a summary of the differences between typed functional languages and unsafe languages:
Kudos to anyone who knows what this means.
Krugman correcting a flaw in an Obama speech about the energy bill. It’s very unfortunate that the president didn’t get this right, since externalities are the whole point behind cap and trade legislation. If he isn’t able to articulate this point consistently and correctly, he won’t (and shouldn’t) be able to convince anyone. Moreover, any bill that comes out of this process that isn’t based on people understanding (and admitting to understanding) externalities will likely be hopelessly flawed.
Imagine a computer stored in a box with a single small hole connecting it to the outside world. We are able to run programs inside the box and receive the results through the hole. In fact, in a sense results are all we can see; if the program makes efficient use of the hardware inside, the size of the hole will prevent us from knowing exactly what went on inside the box (unless we simulate the workings of the box somewhere else, but then the box is useless).
The human brain is a good example. To an outside observer, the hole is speech; other people can’t know what you’re thinking any faster than you can say it. However, speech is also the hole to ourselves. As I write these words, I am only fully aware of them as they appear on the screen in front of me. Until then, I do not consciously know what they are. I can choose to become conscious of them if I say the words to myself internally, but I must slow down in order to do this. The reason is that I am capable of being fully conscious of only one thing at a time, and it is more efficient to be conscious of the words visually on the screen rather than as I “think” of them.
Thus, we have a trade-off between consciousness and efficiency. In order to be fully aware of our thoughts, we must slow them down until they fit through the hole of consciousness. Conscious thought is necessary in order to correct mistakes in our thinking, remember our conclusions, and communicate with others. However, since our brains are internally structured as a massive parallel computer, the only way to use our brains efficiently is to not be aware of what we’re thinking.
This trade-off applies to thoughts of all kinds, and most areas of human endeavor require carefully switching between the different modes. For example, working out a mathematical proof is often said to require intuitive “leaps” of thought. The reasons these appear as leaps is not because they are actually sudden; our brain does not sit around doing nothing and then suddenly have the idea. Rather, our unconscious mind is considering various different avenues of thought in parallel. If an avenue appears fruitful, the unconscious part will make it available to the conscious part, and it seems to “pop into our minds”. Similarly, it is very difficult to consciously try to remember a particular fact. If you let your mind wander, the unconscious part is better able to search around for what we need in a massively parallel fashion, providing us with the answer asynchronously.
Movement is the same. When you dodge a ball thrown at you, your unconscious sees the ball and moves out of the way before there is time to articulate what is happening. Soon afterwards, your brain retroactively explains what happened: “If anyone asks, say we saw the ball coming towards us and dodged.” One of my hobbies is swiveling to fit through doors as they close without touching them. The last time I did this, I had a vague memory of my vision dimming during this motion, almost as if I was passing out. I think this effect was my consciousness temporarily shutting down to avoid interfering with the reflexive motion.
The same trade-off applies to computers and algorithms. Consider the problem of checking a C++ program for type errors. Current compilers do this by running full template instantiation, which generates code for each instance of the template. In other words, the compiler is “conscious” of the entire process. If we all want is the type errors, it would be much faster to use a specially tailored algorithm that checked for errors only, remembering only enough of the results to speech up the rest of the error checking process. It would be even faster if want to know only whether errors exist, not what they are, since the program could forget line numbers and other details. The cool part is that it is possible to combine the different approaches to get the best of all words: we can running error checking before code generation to reduce the latency of reporting errors back to the user, and we can speed up error checking by first running the stripped down yes/no algorithm and reprocessing any portion that has an error. In a decent programming language, all three variants can be generated from the same source using partial specialization.
This last point is very important, since it means that the trade-off between consciousness and efficiency can often be eliminated; we can start with “fully conscious” code (which remembers everything it does) and apply various “forgetfulness” transformations to shift towards the efficiency side. The different versions can be seamlessly interleaved so that it looks from the outside as if the fully conscious version is operating at the speed of the fastest version; the missing detail is recovered only when we need it. This is similar to human vision; any object we look at appears sharp, so we generally imagine that we see with uniformly high detail. What is actually happening is that almost all of our visual field is blurry, with one high resolution point in the center. We can shift this high resolution point to anywhere we wish, so we get the illusion of uniform sharpness for free.
Taking full advantage of this trade-off to speed up programs will require languages that combine low level and high level features and make it easy for the program to inspect and transform its own code. I’ll write more about this later.
When people discuss the future of computers and software, a common worry is that it will become increasingly difficult to produce correct software due to the ongoing surge in complexity. A common joke is to imagine what cars would be like if they were as buggy as software. I believe these fears are groundless, and that they arise from a misunderstanding of the reason why current software is full of bugs.
Modern software contains bugs because bugs aren’t the problem.
The reason we have bugs in computer software is not because writing bug-free code is impossible. Rather, it is because writing bug-free code is expensive. Eliminating all bugs requires enough extra time and money that it isn’t economically advantageous for the vast majority of applications.
In areas where correctness is more critical, we spend more money and get correctness. Hardware is an excellent example of this: a modern CPU has somewhere around a billion transistors, and is usually bug-free. This is because hardware companies spend vast amounts of money on formal verification methods which rigorously check the correctness of circuit designs. It has to be correct in order to work, so we make it correct. This also applies to software in critical areas: when was the last time you heard about a bug in the software for your car?
For now, most software usually doesn’t need to be correct (witness the explosion of dynamically typed scripting languages). As our dependence on software grows, our demands on its correctness will increase, and software will become correct. To do this, we’ll use a combination of model checking, better languages, formal verification, etc. The ideas required to do this have been around for a long time, are constantly improving, and will improve even faster when we need them more and start devoting real resources to them.
I think this shift will happen soon, but the timing depends primarily on the economics of software, not technical issues. As soon as we’re willing to pay extra to get bug-free software, people will start writing it.
It occurred to me that Kurzweil’s singularity might be better characterized as an event horizon. The difference is that a singularity is where something actually infinite happens, while an event horizon is a surface beyond which one can never know anything. An event horizon looks like a singularity from the perspective of a fixed external observer, but from the perspective an observer falling it in always appears to be further down.
Of course, in the case of a black hole anyone who gets near the event horizon is ripped apart by the massive gravitational gradient. Hopefully this won’t happen in the Kurzweil version.
As part of moving, I just got rid of these:
Here’s a quote from a Managing Director at Standard Poor’s, explaining how their years of data on mortgage borrowers could be used to predict the behavior of new types of mortgages (such as those which don’t require proof of a job):
We were able through our analytical process to develop assumptions about what the future would be like for these borrowers. [TAL]
Normally it would be hard to take data from one situation and apply it to another, so it’s a good thing they had an analytical process. Stunning, the guy actually stands behind their decision to rate the securities as AAA. His reasoning is that this crisis is was completely unexpected. I.e., they did a terrible job rating novel financial instruments, and can’t be blamed if something novel happens as a result.
I’m in the middle of listening to this, and had to take a break to rant for a minute. They’re currently interviewing Alister McGrath, who has written two books complaining about Dawkins. Here’s a quote (emphasis mine):
Well I think one very important point here is simply that it’s fairly clear that science is limited in terms of what it can tell us. It is marvelous in clarifying the relationship between entities and forces in the material world, but when it comes to questions of value or meaning it isn’t really quite so good. In fact Dawkins and I disagree on many things but on that point I think we are agreed: science cannot tell us what is ethical, but for me and I sure everyone who is listening I’m sure what is ethical is actually enormously important.
And here’s a Dawkins’ quote from the same program, which I’m going to pull out of context:
Bullshit.
The idea that “science cannot tell us what is ethical” is either absurd or a meaningless tautology. In the quote he explains that science is great at telling us about the material world (or as I like to call it, the world). What he’s arguing is that science is limited to studying physical properties. This is like saying that biology isn’t a very good way of studying general relativity; if he’s trying to separate science from logic and reason, his statement is a meaningless tautology.
Or not: maybe he’s really saying that logic and reason can’t determine what is ethical. Has he heard of philosophy? What does he think all those famous theologians spend their time doing? They start with some basic premises, and working out most of the rest using reason. Arguing that science has nothing to say about this subject is absurd.
What can science say about the basic premises? An endless amount: since logic and science help us explore the consequences of basic assumptions, they help us judge whether the basic assumptions are reasonable. For example: why is being nice to people more ethical than wandering around killing everyone in sight? Because if you wander around killing everyone in sight, people will be frightened and miserable, and will probably come after you and stop you. It’s just not a sustainable way for the majority of people in a society to behave.
End rant. Back to packing…
