Rob Hope

As a writer and reviewer of documents, I’ve spent a lot of time considering how I would want to leverage AI tools to improve my writing. In most cases, I’ve observed the development of models that are similar to what something like Grammarly might provide. They can correct grammar, make suggestions on sentence structure, and potentially point out complex or unconfident words.

There is an AI improvement tool available within the portal I use to write my blog posts. I agree with only about half of what it suggests. As an immediate example, “provide” is not a complex word, but my AI suggestion bot thinks it is. I’m a linear writer by and large; I still outline my thoughts before I start, because I want to be sure I progress from idea to idea in a way that is easy to follow. Sometimes that results in long sentences, but when I use them, it’s with a purpose in mind. I’m very deliberate about my choices when I write.

And that’s where I tend to disagree with most modern writing agents when it comes to providing writing feedback. They can correct mechanics, possibly better and more consistently than I can. But where they miss is in language tone, word choice, elegance of phrase, use of techniques like alliteration even in prose, and other more subjective applications of writing skill.

They lose the uniqueness of the human perspective.

One of the topics that came up often as I reviewed documents at Amazon is how do we distill each reviewer’s unique approach to analysis into models that we can then deploy, and the concept I landed on was what I called “micro agents”. Rather than incorporate everything into a single large model that would then have to make judgments about which feedback to apply, I thought it would be more effective to be able to train a model how I, Rob, would review a document. I would then train another model how another reviewer would review the same document, because the feedback would be different. If I could come up with 10 or 20 models containing each reviewer’s “personalities”, and then deploy those, an author could then select which reviewer or reviewers they would like to apply.

There are several advantages to this approach.

First, the author could target tonally consistent perspectives. I don’t mind complex words, so I’d prefer to get feedback from someone (or something) that likewise is OK with complex words. And as a developer of a model, I don’t want to introduce that feedback loop into a model of another reviewer who has a different perspective on complex words.

Second, the author could leverage feedback with different perspectives from a consistent source without having an AI filter that perspective down or summarize options that are contradictory. I’ve literally had my AI arguing with itself at times as I’ve been writing content because it can’t maintain tonal consistency.

And third, the models could learn independently across many different iterations of different documents without all of them ending up at the same conclusion point. While there is an element of a selection bias by allowing the author to pick specific “experts” to give them advice, that also means that the feedback loop is relevant specifically to the expertise the model has been trained in.

In practice, I don’t want an all knowing model telling me what to do against a filtered set of options with a potential learning bias. I want to seek out the advice of experts at the thing I am doing who can be very, very good at the analysis I require; if I can’t get to them personally, then a model that thinks like them is the next best thing.

I don’t want my writing to end up sounding like everyone else’s because I used AI.

The same thing could apply to my composition of music. In a previous post, I talked about my interactions with ChatGPT as I composed my latest work, a Baroque style symphony. Imagine a composing world where you could pick two or three specific composers from a list and get feedback on how they specifically might approach a problem rather than a generalized answer. Several times I found myself disregarding feedback because it was tonally out of place. Several times I found myself arguing with ChatGPT about specific applications of things, and the answers, while thorough and grounded in theory, didn’t always tell me why they were being suggested or even if they aligned with the style of music I was writing.

As part of my interest in writing, I’ll be exploring if I can train an AI agent to review documents like I do, including analyzing where my approach differs from conventional wisdom. It will be interesting to see where that lands.

Happy writing!