I spent the week in Cape Town. I always enjoy my time there.
On the drive from Knysna, the N2 passes through massive tracts of farmland as far as the eye can see, up until you see the vast mountain ranges that guard the route to the Karoo. First, the Outeniqua Mountain range as you drive past George, and going onto the Langeberg mountain range as you pass through Swellendam.
It’s canola season for the farmers as their fields are bright yellow with canola flowers. I am amazed at the efficiency and reminded of how absolutely nothing is wasted out there.
I listened to a fascinating podcast of Lex Friedman talking to Elon Musk about his companies. They covered a vast array of topics, but one idea stuck out in Musk talking about how one goes about building the best AI system in the world.
Let’s say we have a formula 1 race. What matters more, the car or the driver? They both matter. But if the car is not fast, let’s say half the horse power of the competitors, the best driver will still lose. And if the car is twice the horse power, then probably even a mediocre driver will still win.
The training computer used in AI is kind of like the engine or ‘horsepower’. How efficiently you do the inference or use case of the AI and what unique access to the data you have is the human element (the driver).
So, what does it mean to win the AI race? Compute power (the engine) is the decisive factor. If your compute is slower, you are going to have worse AI, and you’re going to lose. It’s as simple as that. No-wonder companies are investing hundreds of billions into this “horsepower”, with firms like NVIDIA benefiting from this surge in demand. Just look at their shareprice over the past 5 years – up over 2 500%
Naturally, Musk is going big on compute.
Another aspect I found fascinating was Elons description of a computer’s relationship to humans, and our ability to communicate with it.
With the massive computational ability and sheer power of what has gone into these machines, they are able to perform trillions of operations every second. Musk says that for a computer, talking to us is like talking to a slow-moving tree. And this creates a barrier in their ability to understand us.
What does he mean?
If you have computers that are doing trillions of instructions per second, and when a second goes by between each of our keystrokes on our laptops or phones, that’s a trillion things the computer could have done while it was waiting for us. That’s an eternity.
This is something he’s trying to solve through his company Neuralink. I’m not quite sure how we get around this, or whether it’s even a problem (besides frustrating the machines). But it gives us a sense of the power of these machines and their capabilities—we aren’t using a fraction of what they can do. We are just trees to them.
It’s going to be fascinating to see what he does in the future.
