You can always pay more to wait less
This week, one of my programs ran out of memory and crashed.
At that point, I had two options:
-
Rewrite it to use less memory, which would make it slower.
Buy more compute to keep it fast.
That led me to these questions:
-
How long am I willing to wait for that program to finish?
-
How much am I willing to pay for that program to finish?
In that case, the answer was obvious. But that's not always true.
And I tend to think that the art of programming AI systems lies in the ability to weigh the trade-offs between waiting and paying.
Because for some kinds of problems that AI can solve, you can always pay more to wait less.
So deciding where to stop is the real work.