Recent news of the CEO of Shopify blaming inefficient work on "AI Slop Grenades" reminds me of a conversation I had that went something like this:
Our company makes a SaaS product, and our customers overwhelmingly love it, but our investors just made a stink over how several other startups recently went from sure bets to closing up shop because BigCo just launched a part of their AI suite into the same vertical. This means no one needs to pay for their service anymore, because they can just have BigCo’s AI agent do it instead. Long story short, we have to put AI in our product, and start using AI in our product development process, because otherwise we’re doomed!
The math of that logic didn’t add up, but I wasn’t in a position to push back. Over the next several months, they pushed AI usage in every process, and proposed AI to solve every sort of problem. Customers had issues figuring out if they could use a vendor; replace browsing vendors with an AI chat interface! Set an unreasonable delivery date; just have someone vibe code and push it asap! Don’t have enough people to do the work; write up an AI agent that handles all the project management, and then also make it do code reviews, and build prototypes, and…
While this may sound like a tech capitalist fever dream, this is also all too common at the moment. “Slap AI into it” has become the modus operandi of the “move fast; break stuff” set. It’s alluring, because anyone can do things, and sometimes those things work, and paying for a metric tonne of AI tokens apparently nets out to be cheaper than hiring—or keeping—skilled professionals. In the above example, the company saw super-senior ICs and leadership members leave suddenly. They saw their customer satisfaction plummet and their support queue explode. The product became wildly inconsistent as the design and development process morphed into Support question in Slack -> AI agent makes a ticket -> AI triages ticket -> developer picks it up and has an agent code up the AI derived solution, has an AI agent test it, an AI agent review it, and then they push it to production. That may seem like an ideal amount of response to customer feedback, but to see where this falls apart, you need to dive a little deeper into what that implies, and what is missing.
Customer feedback
As a designer focused on meeting the needs of people affected by whatever product I’m working on, I know how crucial it is to listen to your customers. Feedback is the backbone of UX design. However, when the feedback fueling change is narrow and not studied and validated, it becomes the spitball idea of someone who has not had the chance to consider and fully articulate the problem. “Show this piece of data here!” can seem like a reasonable request, backed by that customer’s very real experience. But without deeper scrutiny and discussion around the issue, the core problem is not discovered, and the solution adopted either only eases the frustration of a limited set of people, or more often, causes unintended new issues. With enough of these special snowflake fixes, an avalanche can quickly form. Discovery, study, discussion—all with multiple sources and led by someone experienced in the nuances of finding the real problems behind feedback—are key to avoiding that storm. Assuming your support agent will translate or supplement the data of the support ticket well enough that the core problem is identified, even with an AI agent, is a big ask of someone likely untrained and unqualified to do so.
Ticket this
So let’s take a giant grain of salt and assume the support process did an exceptional job—yes, it can happen—and they have the data to prompt an agent to write a ticket for something to be changed in the product. Or even a skilled product manager was involved, and they’re including supplemental data. Awesome. But instead of sitting with the problem, or having a discussion with engineering and design, they instead feed the support data, the Slack conversations, and random app tracking data into their favorite AI agent, and say “write a ticket to solve this” or “write a Product Requirements Document (PRD) for this.” The LLM does what it does—I’ll let you hold your own opinions on whether it’s adept or just computer hallucinations—and spits out a PRD and a ticket. In the space of “move fast; break things” the results may only get a cursory skim, and in the best case, a few people review it and refine it. In the PRD case, this is more likely; a ticket alone often won’t get many eyes on it. But in either case, more and more trust is being placed in what the AI documented as being both accurate and the correct course of action. The old meetings discussing the nuances and considering pathways have vanished, replaced by a few notes made in the Notion or Google Doc, because faster!
Prioritize this
Admittedly, this is a grey area. Sometimes they’re working with a ticket to change a feature to fix a support issue, and the person in Support likely marked the ticket as High or even Urgent. More and more, folks are opting to let the AI agents prioritize and delegate tasks. If person X tends to mark their tickets as urgent, and person Y just says Medium or High, without actual review, person X’s ticket will routinely get done first. All it takes is person Y realizing this and switching to all Urgent to suddenly make the ticket prioritization moot, and AI is more than happy to silently fail and never call out the logic it uses to prioritize tickets.
Straight to a developer
Sometimes, all a good ticket needs is a developer to read it and make a small fix. This is fine, and it’s a case where this AI-heavy workflow tends to yield positive results. But the input in this case is more likely to be “My report is showing inconsistent data, I think it’s a rounding error” than it is a case of “this page is confusing to me, so can you add this specific data point to make it useful?” In the latter case, even if a product manager gets involved, and the PRD they created is decent, entirely skipping design input is, at best, going to contribute to UX debt—the slow accumulation of of inconsistent design, UI and interaction elements that make applications and services more frustrating and difficult to use. Skipping design used to lead to what we would call a “developer special”, where the result was rough around the edges, but probably functional. These were considered Bandaid releases, something to quickly fix a problem but that would probably need more effort sooner than later.
With the introduction of generative AI, though, it has become far more common to see a developer or PM feed that ticket or PRD into something like Claude Design or even straight into Claude code or similar agents to quickly dream up a UI prototype. Maybe the developer and the PM discuss the options that result, maybe the developer just points their coding agent at it and says “build that UI in the repo” while they write the code to wire it together. I’ll circle back to the frustrations and guaranteed UX debt inherent in using AI generated designs, but anyone spending more than 2 minutes on LinkedIn with connections that are experienced code wizards will tell you that the code these agents put out is fragile garbage, the kind of stuff where you’re not quite sure why it’s working, and might entirely stop working if you look at it too long. A good AI coding agent can help a talented developer avoid repetitive tasks like boilerplate stuff and a known and often reused function, but in the “move fast, break things” world, the talented devs are changing careers and starting farms, and the vibe coders with little functional understanding and even lower wages are taking their places. Yikes.
Don’t worry… we test and review code!
In the example company, the leader had said “we let our customers do the QA work.” If you know what a QA (Quality Assurance) engineer does, that statement should horrify you, but it’s an attitude that predates the AI bubble. It comes from the capitalist logic of “why hire 2 when 1 can do?” and assumes that the engineer that codes the feature is the best person to test the feature. I can hear QAs and SDETs screaming in the back. Okay, well what if we minimized the inherent bias by having AI do the testing? This doesn’t sound far off of what QAs call Automated Testing, except it abstracts out the human eyes double checking, and ignores manual testing (where the real life bugs are often discovered). Not great, but better than nothing.
Assuming those tests are passed, the engineer then creates a Pull Request (PR) to have their new code reviewed and merged into the production code, awaiting the next deployment of the app. Finally, some human eyes to scrutinize the code. Except a PR just shows the code, not the functional app and UI. A good PR review should include this, done by the reviewer, but that takes time, and we have to move fast. How do we move faster? We skip steps like that. How do we move fastest? Have an AI agent review the code. And you know what? It catches a lot of bugs and formatting errors… that another AI agent created. But it does nothing in reviewing the actual UI experience, because, well… it’s code, not a human. Assuming the review passes, the code is merged and will likely be pushed to production shortly thereafter.
It’s code, not a human.
Let’s sit with that concept for a minute. Large language models (LLMs) that drive most of these agents we call AI are really quite good at mimicking human writing and speech. When you train an algorithm with an endless stream of human writing, that’s bound to happen. It can be easy to find yourself viewing your work with an AI agent like any other chat with a coworker. Anthropic even calls main workplace app for Claude “Cowork.” Factor in the bias toward making the user happy and validated, and it’s easy to let your guard down, even with the ever present “this is AI, it can make mistakes” mouse text below your input box. But consider what you’re placing your trust in—a programmatically sycophantic chat window that bases its responses on probability and the bias toward any implied direction. That’s problematic enough when it comes down to generating ideas toward a solution; it’s downright scary when you’re trusting it to perform most of the steps in making your product.
I’ve often remarked that working with AI design tools is like coworking with the most junior designer you can imagine, fused with the confidence of the head of sales. Imagine your nephew who’s “good with Photoshop” (to use the parlance of my day), and really wants to impress you, but also doesn’t pay close attention to details or anything implied, and who is dead certain that whatever they show you is 99% perfect. So, as a designer with [redacted because ageism] years experience, you give this kid all the data and documentation you have, access to the corporate intranet, and ask the kid to bring back 2-3 variants. Your nephew is young and fast, and you’re surprised when he says he’s done and asks if you want him to add the Figma link to the ticket and mark it Ready for Engineering. Whoa, slow down kiddo. You look at the results, and at first glance, they are quite slick and polished. You like option 3, so you start doing a more critical review. You realize he didn’t use the font from the design system. Okay, quick fix. But wait, the typography doesn’t use the established design tokens from the design system either… these are just arbitrary values. Okay, quick fix. No, you didn’t actually use the tokens in the Figma file, just the values that correspond, which will make the engineering team need to look up the corresponding tokens. And even that isn’t consistent, nor is the spacing. We have tokens for that. Please use them. Okay, quick fix. Wait, why did you detach the components… wait, did you just remake the components whole cloth? We have a linked component library for that, and now they’re unbound and won’t get updated. And what are these rounded pill-shaped chips, and why are they using a font smaller than our scale? Okay, quick fix. Umm… yeah, that’d didn’t use the components, still. And why aren’t you using the icon set as defined and included in the components library? Wait, you think you did use them? You took a screenshot and determined they are correct? No, I can tell you they are not using the right icons… oh, so you say you don’t have access to the paid version of those icons in your toolkit, so even though I can point you to the literal node URL of the icon to use, you’ll just keep using a rough approximation and tell me it’s correct? Ad infinitum…
It's a very real example of arguing with AI over the number of E’s in seventeen.
It’s incredibly frustrating doing the fine toothed combing required for a good—structurally, systematically, and yes, aesthetically—design, especially given that something driven by code should be good at the first two parts, given access to a solid design system. But even more frustrating is that all this work assumes that this is the actual solution to the problem. When penny-pinching or technophilliac business people, rarely with a lick of design training or understanding, see the first results the AI nephew produced, they don’t dig any deeper than “ooo shiny” and “I like option B.” And that’s actually fine that they’re not design savvy—that’s why they employ trained, skilled designers to make the right decisions and present them with either options or well-supported reasoning behind the chosen design. The best business leaders I’ve worked with have trusted the product and design folk to make the smart calls, the same way they trust the engineers to not write shitty code. But the AI nephew scratches that itch of both fast and cheap, and they are enthralled.
Now, you may say Cheap!? These AI tools work on a pay-to-play basis. They are so pricey!—and you would be right. The AI business model is based on the concept of AI being required, like a utility. And recent earnings calls have shown that this model is not paying out. What I mean by cheap is the business logic that compares an overall cost for AI tokens (often incomplete or handwavy) to the combined costs of salary, benefits, equipment, and management drag that real people constitute. There’s a reason they refer to employees as cost centers in HR and Operations. The allure of a minimal staff, the braggable luster of using the latest tech, and the idea that “growth always wins” can make a businessperson be shortsighted. But eventually, moving fast and not building a level of human introspection into your process ends up breaking things, like your customer retention and satisfaction, your product stability, your brand image, and ultimately, your bottom line. The tech billionaires have the luxury to ignore this—for the time being—because they aren’t trying to solve your business problem. By getting you hooked and convinced that AI is the necessary solution, you’re solving their problem.
And don’t get me started on what they’re doing with all the business data you’re feeding into these systems.