First, an editorial note: After a long hiatus, I found the space to write again and as part of this restart, I’ve re-read all the older posts. They certainly feel like they haul from a different era. To be fair, they are still very much relevant. Many of the challenges discussed in the posts are – frustratingly – timeless. But it would be a mistake not to acknowledge that something profound is afoot, as “general purpose, on-demand cognition” technology makes its way through, well, everything, least of all technology organizations. Say what you want about AI’s promise, perils and its ability to deliver, the discourse is real. True to fashion, I’ll aim to approach the topic with curiosity and openness, but I’ll be the first to acknowledge the dangers of writing about something so dynamic, ever-changing, and yes, noisy. Rather than be defeatist, I’ll be prepared – and will ask you to expect – significant revisions to many of the points I make along this journey. Worst case we’ll be meandering, backtracking, and may emerge not far from where we started, but sometimes it is that very journey that matters more than the ultimate destination.

Writing About AI

Writing about AI is fraught with challenges. First, there is the constant change. The underlying foundation models are undergoing what seems to be robust exponential improvement. The wrapper applications (e.g. Claude Desktop) are being updated daily, attempting to create the best metaphors and user interface models to maximize the LLMs’ effectiveness. The harnesses (what the LLMs have access to, their “eyes and hands”) are increasing in reach and sophistication. And not to mention new paradigms (Agentic AI! Open Claw!) which constantly reset our mental model of how we should interact with this almost-too-general-purpose technology.

As a technology, generative AI feels more “alien” than other foundational technologies mankind has experienced. Granted, I wasn’t around when electricity transformed the world, and perhaps those first experiencing the effects of electricity have had the same sense of uncanny unfamiliarity, magic, and awe bordering with terror. But AI’s “jagged frontier” of capabilities (seemingly similar tasks show wildly different success rates), incidence of hallucinations, and, let’s be honest, the fact that we don’t really know how it all works (and maybe never will), all add up to something quite difficult to digest.

Worse, I have never seen so much disagreement about any technology, which adds to the challenge; disagreement not just about its future, but also its present state. You will hear “AGI is already here” stated by some as confidently as “AI is all slop and shallow pattern matching” is claimed by others. For every Tomasz Tungus there is a Gary Marcus. Bloggers talk about the mind-blowing productivity gains, yet most seem to only ask AI to build their long-ago-shelved pet projects. Candidly, I think these paradoxes speak to our fundamental discomfort with what generative AI attempts to be: an encapsulation of cognition, our most prized ability, into a function call.

Finally – and this may be unique to AI, its general-purposefulness and its potential for monetization, but it may also be the sign of the times – the sheer amount of noise and AI-peddling is remarkable. Of course Salesforce and Microsoft are going to talk about their teams’ massive productivity gains thanks to AI – these companies make a lot of revenue selling AI products and features! That’s not to say that we shouldn’t trust anything these AI vendors are saying; but a massive grain of salt is recommended.

To Study, or Not To Study

All this exponential change, the “surface area irregularities”, the philosophical divide and the noise mean that AI use is hard to study, and best practices hard to come by.

And the desire to study it and to extract best practices is well-justified. If you are like me, you are most likely amazed – and continue to be amazed – by the technology, especially for applications which it seems to be perfect for, such as software development. I invited Claude Code to join me in the development and maintenance of a personal productivity project of mine, which started during Covid and which is the one application that I use hundreds of times daily. I took my wish-list of over 50 items that needed fixing or changing and force-ranked them based on my expectations of what Claude Code should be able to do. I expected it to tackle the first 3-4 without much help, another 4-5 with some guidance, and would have been pleasantly surprised if it did okay on the further 5-6. Claude Code blew past these (in retrospect, arbitrary) markers. There seems to be no limit to its ability to problem-solve with code. And it’s not just software development – I’ve been using AI as my personal and professional “Advisor / Chief of Staff” and the quality of my thinking and decision-making has increased significantly. AI keeps me honest about my blind spots, helps spot patterns and connections across a wide range of material, offers thoughtful critiques and robust plans of action. It’s not as good as the best human for each “hat” I give it, but it’s easily in the top 10th percentile, wearing all those hats at once.

We see the wonders AI does, and extrapolate into the future, and can’t help feeling that in the near future, as AI matures and gets incorporated into our tools, habits, practices, and organizations, the world will look very different. But how different? How near of a future? How should it get integrated? Your guess is literally as good as mine. Oh, and humans are notoriously bad at extrapolating from exponential curves.

But rather than throw up our hands and call the problem intractable (which I think would be enormously wasteful, if not outright irresponsible), we should be able to do what LLMs are very good at – identify patterns and themes, and look for invariants. Let me propose a few of these invariants, a result of observations and learnings from the last several years of my AI journey.

What We Know

LLMs are already good enough. We don’t need to wait for GPT 8.1. But, LLMs alone are not enough. We now know that it’s the integrations, and the context/memory, that are the new bottleneck. This is true at a superficial level (for example, access to data, or the ability to navigate the online world as effortlessly as you and I do), but more so, at a deep level (the extent to which LLMs can integrate into our workflows; the ability for LLMs to process and compact memory – we know just throwing a ton of data into the context window doesn’t work because of context rot, and is quite expensive).

In fact, a unique property of AI is that its value is proportional to the depth of its integration into our processes, foundations, and lives. Most of AI’s usage in the world today is what I would call “shallow” use. It’s the “AI as a one-shot question-and-answer chatbot” pattern – people asking factual questions, asking for AI to draw an image, or recommend a place. It’s the easiest way to engage with AI to start with and doesn’t require much investment (uploading of context, memory, integrating with tools and data sources), but using it that way can by definition only provide fairly superficial value to the user.

It’s this property that, in my view, makes it difficult to copy-paste AI benefits across companies. True, multiple companies have been able to package AI into a well-defined service that companies can plug into their ecosystems, but for the kind of transformational effects that we all hope for – or expect from – AI, there are no “package solutions”. In a way, the “SaaS gold rush” of the 2010s and early 2020s spoilt us, expecting there to be a plug-and-play solution for the various tasks being performed in an enterprise. In my view, AI lends itself more to bespoke solutions that uniquely fit each company, more like jewelry than a hammer.

That’s why I’m not a fan of comparisons of AI to the advent of the Internet or mobile computing. A mobile device has very high affordance – you literally point and tap, as everyone who had a two-year-old play with a touchscreen can attest to. And the Internet took advantage of the preceding decades of the “human-computer interface revolution". True, the Internet transformed business models, making zero marginal costs possible in a vast array of scenarios, but as a technology it is actually quite intuitive.

Because of this bespoke nature of AI application, I believe another invariant is that in order to extract true, deep value out of this technology, teams need to have the space to experiment and have the permission to fail. In a way, the job of integrating AI is a true R&D function. It’s a creative endeavor, where learnings from mistakes might be more valuable, on balance, than learnings from successes. Dissemination of ideas, and building on one another’s hypotheses and building blocks is paramount. Organizations that don’t create that space for teams, or that demand specific improvement targets upfront, are missing the point. Similarly, individuals who aren’t curious, don’t experiment and don’t talk to other teams, building upon their solutions will be unlikely to leverage AI as well.

What’s Next

This post has served an unenviable purpose as being the first in the series about organizations’ (and my) AI journeys. Of course, a conceptual introduction is merely the top of the iceberg. In subsequent posts I’ll dig deeper into ways in which organizations could approach integrating AI into their day-to-day. But I’ve hopefully served the important word of caution: while there is a lot we can learn from each other, don’t expect a turnkey recipe for success.