Notes

I felt I should capture ‘I run slow ’ here, just to remind my future self (thanks Phil Daian for sharing)

I came across this post from Amit Sahai. The most important point he makes is to invest into the collective human understanding of new ideas discovered by AI or a human-AI collaboration. It’ll be a waste if we fail to use the new results to develop a deeper understanding of the world. And it’ll very bad, perhaps even lethal, to live in a world surrounded by systems that no human understands.

While I agree with Professor Sahai, I have one objection to his post. It assumes that AI systems have demonstrated self-imporovement beyond the scope of human produced knowledge. On other side, I remain optimistic that human-AI collaboration will create new wave of progress. But this progress will be steered by humans, and they will understand & control it. Likely this progress will eventually lead to AI systems that can make leaps.

KK’s essay is enlightning. His description that capabilities of LLMs is an emergent property of tranformation of millions of sources into the latent space is sharp.

I also find the comment ’the latent space is the new commons’ interesting. Needless to say, LLMs are incredibly powerful. It’s strategically important to saturate LLMs, both for a nation that wants to remain relevant and humanity to progress at a rapid pace. And it’s a shame that the frontier models are not universally available (even when one is willing to pay the cost). I hope governments recognize this and direct substantial portion of tax payer’s money to (a) develop public intelligence, and (b) have LLMs saturate in its population.

Mathematicians are realising that today it’s no longer true that solution to a problem contributes directly to the goals of the mathematical community. For a long time, training as a mathematician with an eye for solving big problems required developing a good understanding of the field. It required asking good questions, develop new abstractions, distill new insights, and discover new ideas. Progress made on hard problems was a good proxy to measure progress in the field, and our understading thereof. This is no-longer true. LLMs, trained on the vast human produced mathematical knowledge, can produce solutions to hard mathematical problems without contributing anything in return to the shared understanding. This is concerning because (1) it reduces the incentive for a human mathematician to undergo years of training to develop an understanding of the field. Why should they fight the uphill battle of understanding, while pointing the model to the right problem seems to acheive the outcome rewarded by the system. And (2) if LLMs fail to develop understanding, while humans have little incentive to, then the progress will plateau once the steam in existing human knowledge runs out. The mathematical community is waking up to this, and their immediate remedy is to award development of understanding vs, what traditionally existed, awarding solutions to a problem. I find this a good development.

I understand that (2) is invalid if RSI succeeds, and I hope it does. But there’s something else, much deeper, that I’d like to ask: why practice mathematics? More generally, why engage in any kind of intellectual work? Is there something uniquely human about expanding our understanding of the world, the cosmos? Even after we’ve machines that can do this much faster than us, is our engagement still justified? hmmmmm…I think so.

However, I think that future is a bit afar (or maybe not). Today’s AI has moved human’s work to a higher abstraction. To theorising, asking questions, imagining. Mysticism. Dreaming..the things that are uniquely human and that have caused our civilization to become what it’s today. We shall remain at the throne, atleast for a little while. Until one of the geniuses among us figures how to make machines creative. Have it dream. Meanwhile the other genius fuses themselves with the creative machine, in the next phase of our evolution.

Read more at post by Grant Sanderson, and the latter at mathandai.org

This article from pocoo is closely related to some of our product thinking on PAC. It’s true that LLMs are incredibly useful in pursuing ‘random’ interests, or long-due ideas/experiments, not only for hackers but, increasingly, for the general crowd. Many of the interests are nothing new. It’s likely that someone has already tried out something similar before. And so, a LLM gradually nudges the user towards a suitable solution. A unsurprising consequence of this is that the end results often overlap.

Some interests are indeed new; an open-ended curiousity. They have no prior art. In such cases LLMs are still useful to find relations across & take inspirations from what exist today. But the end results often differ, and, sometimes, differ by a wide margin irrespective of the overlap in the users .

Therefore, it’s not surprising that for many LLMs are the first touch to explore any idea, experiment, or question. And LLMs will increasingly become an rich log of an individuals latest interest & ideas.

This opens a new dimension to social life. I’m hopeful, that due to the peculiar nature of this medium that compels the user to fill it with rich information, the social connections made & new communities started from this rich context will be far more fruitful, satisfying & productive than the ones that exist today. This is of importance as our existing social networks give up to AI slop.