I came across this post from Amit Sahai. The most important point he makes is to invest into the collective human understanding of new ideas discovered by AI or a human-AI collaboration. It’ll be a waste if we fail to use the new results to develop a deeper understanding of the world. And it’ll very bad, perhaps even lethal, to live in a world surrounded by systems that no human understands.
While I agree with Professor Sahai, I have one objection to his post. It assumes that AI systems have demonstrated self-imporovement beyond the scope of human produced knowledge. On other side, I remain optimistic that human-AI collaboration will create new wave of progress. But this progress will be steered by humans, and they will understand & control it. Likely this progress will eventually lead to AI systems that can make leaps.