When AI Reviews Design

Welcome to The Daily Sprint, where today is a great day to design.

This podcast is for product designers who want to go beyond pixel pushing and lead by designing for value into their products one day at a time.

Today we'll be talking about AI and design reviews.

What are they?

Should you use them and what are the alternatives?

I'm Darrell Estabrook, 30 years into UX product design. Yes, it's before intelligence was artificial, and you had to save your work manually every 5 minutes.

How far we've come.

I'm also the founder of designing a platform for product designers who want to design with a why.

I coach designers into leaders through real-time, interactive product specific guidance.

Find out more and get on board with a free newsletter at designy.com. That's “design” with a ”y” dot com.

So welcome.

Thanks for tuning in to episode five.

If you keep coming back, there's plenty of more to give.

So this podcast is called the Daily Sprint.

And while I would love for this to be daily, it's not the intent per se, it's really to frame your design mindset.

So the thing about it, daily, right, you're only given one day to make design decisions.

You can't make them yesterday and tomorrow isn't here yet.

So set up those meetings, try a new technique.

Take an action today.

And then the sprint part is to time box your design decisions.

It's a quick run, right?

A sprint.

This is not a marathon.

Break down the things you have into smaller bits.

Be inventive.

I used to say laziness is my greatest motivator as far as trying to get the same work done in less complexity.

So the impact of your choices are going to come later, but every day is up to you.

So here we are.

And everything is a design decision.

Have you ever thought about that?

Think about this, what did you choose to drink this morning?

I don't know, if you have to choose between coffee or orange juice.

Some people might call that a preference, but really, if you think about it, you're picking one over the other for a purpose.

Right?

My wife is usually cold.

So she's always going to pick the hot coffee.

I guess unless it's in the summertime, then iced coffee works, but that's a different purpose.

And there's a 1000 reasons why people might choose what they're choosing, but they're all the result of trying to achieve an intended outcome.

Right?

And so you have this coffee, because you're cold.

So what if the coffee gets room temperature?

Well, now that outcome isn't achieved anymore and you've got a new problem to solve.

And that's designing.

You've got a design for that issue.

So you might not think this way about every decision, but yet it's still there.

You might not want to think this detailed about every aspect of your day today.

Should I sit here or sit there?

But you might make more deliberate decisions if you did.

But the orange juice thing is a silly illustration, but it really points to what we do in product design.

Every design decision we make has to be there for a purpose.

It's like if you said, I like putting borders around card elements, right?

That's effectively arbitrary.

Because you can have someone else look at that in like a stakeholder and they'll say, I don't like borders around cards.

And you might think, but I'm the designer.

I always put borders around cards.

But if you step back, you think the real question is, what's the purpose for the borders?

And rephrase it to outcomes, right?

What's the intended outcome of putting borders around cars?

When you say, I like, it's a purpose, but it's not a very strong purpose, because it can get overridden by someone else.

But if the outcome is to enhance visibility of that card, then that's a great reason, that's very strong, and it's one of many possibilities for for doing that.

So you can always use purpose as your litmus test to vet your design decisions or on the suggestions of stakeholders.

You know, what is the outcome we're trying to achieve?

And that kind of brings us to design reviews.

This is where design decisions meet purpose in the real world.

A design review should always answer the question, how strongly does the design fulfill the purpose?

And when you do that, there's really no room for I like or I don't like.

So this idea of giving feedback.

It's one of the greatest skills a designer can learn.

I think everyone should have this on the top of their list.

It might be the 2nd most important skill under the actual ability of making design decisions, and you might stretch it to say it's the number one skill, since you should be critiquing your own work as you design it.

She'll always be thinking, why did I put that border there?

Why am I making this color change?

You might do it because you like it at first, right?

Or it feels right in the composition or there's some other thing.

But if you do it that way, then you need to back up and say, okay, but what is its function?

Like, why is that there?

And it could be that it's, you know, enhances the brand, right?

That's a purpose.

That's an intended outcome.

It doesn't always have to be a metric.

But it does have to be explained.

I have a quick course on this.

It's free and we'll talk about it later.

It's at academy.designey.com.

But the whole deep dive into giving feedback, and that's a, that's another topic for another day.

But today we're going to look at the AI design reviews and consider them.

Is it worth doing?

Are they better than live?

I've got 3 instances where I've kind of vetted this out.

It's very interesting.

So one of the tools that I saw come across was UXPilot, and they just released this design review feature.

So if you haven't used it and I don't use it on a regular basis, there's more to explore in this tool.

But essentially, it's an AI screen generator, and it works a bit like figma, and you can prompt new designs.

I like to think of it as Photoshop's generative fill feature, except you're doing this with product design.

And there's a lot more that you can do with it, but I had it generate a dashboard for a fictional manufacturing facility.

The plant manager is trying to look at the process and everything like that.

And I just did it from a prompt.

So kind of elaborate prompt, but essentially just as much of the requirements as I could get in there.

And it created 4 variations.

So I took one of those variations and I ran it through the design review feature.

So how about that?

You're consuming its own product.

And it was a very comprehensive report.

It had overview section, the scorecard, top issues, quick wins, a visual style adjustment, and visual notes, and there were some references as well.

Like hyperlinks to where a lot of this came from.

And it does that in seconds.

I mean, under a minute, I think.

It went by really quick.

So the big question is, is it useful?

And the net net is mostly no.

There's some good little bits in there, but when I look at it overall, I'm thinking, how am I going to take this and turn it around and apply it?

And so we'll break it down a little bit.

It's a lot about what the design is, but not about how effective it is.

And that's really what we're going for with the review.

There's things to clean up and there's things to adjust.

But ultimately, it doesn't matter, again, with, unless you have the purpose laid out.

And so this says strong, high-level awareness, the dashboard effectively uses a top-down information architecture, placing high-level KPIs at the top to give a plant manager an immediate pulse of the operation.

That's good.

I mean, that's that is what it is, but it's what I was setting out to do.

It does say good use of status indicators.

Colored code is status, dots and badges provide quick, attentive cues.

Okay, well, but that is what it is.

Intended device desktop.

Well, that is what it is.

It's not helping me.

There was one that was interesting. Information density risk.

And it says, well, comprehensive, the production stage status and critical raw materials are quite dense. Plant managing needs to identify exceptions quickly.

So that's all it says on that.

It doesn't say how to fix it or what to do or suggestions, like it's it's just stating a thing.

And that's just problematic in a very static review like this.

The other thing, there's a scorecard that comes after it.

Again, very interesting categories like usability, accessibility, visual hierarchy, layout in spacing, typography, all of that.

And it has a score in notes, and the score, it's out of five, but like usability four.

And I don't know what that is.

I mean, it's 80%, 4 out of five.

But it doesn't really tell me anything.

Um, notes, logical flow, navigation is clear and purposeful for the role.

The problem here is, I don't know what will make it a five.

Because I want to make it a 5 if it's being scored a one to five.

But one to five, what does that mean?

I mean, I get it, but it makes it, it does make it difficult to take action on it.

It has top issues and these are interesting in that there's a lot of accessibility notes.

So, just contrast issues.

What's interesting is this is, unlike other accessibility tests that are out there where they're pretty comprehensive.

In fact, they're every element on the page. Like this one severity, critical, insufficient text contrast on status badges.

So severity, critical, evidence, light green text, and the white text inside, and that's what you're looking at.

And then why it matters, which is kind of interesting.

It says for a plant manager, viewing this on a factory floor tablet or monitor with glare, low contrast text is nearly impossible to read.

And then the fix.

Dark in the green text, and to a deeper shade, and it gives an example hex code, and ensure all systems operational text uses a high contrast dark color or a much darker color.

Again, it's good direction.

Uh, and there's more like that, like change this, you know, alter that, uh, even some of the quick wins, uh, and visual and styling adjustments.

It's like change these.

I've got bold the primary KPI values, increased contrast on online labels, add tool tips to bars.

So we have a good laundry list of things to look at.

But it's, they're limited and, but I'm going in a conversation where there's still not a lot of context and these specific fixes, some of them are going to be mutually exclusive.

I don't have to test it to know.

I just, there's a lot of things that usually when you fix something else changes in the design that, that, it throws another purpose off.

So it's still some experimentation.

So directionally, it might be good, but as I fix this and now you're going to have a great design, it's far, far from that and some of it is just confusing.

So another one that I looked at was a figma plug-in.

There's a number of these out there, but this one, you click a frame that has your design in it and send it for review, and then comments appear.

In this case, it didn't appear on specific elements within the screen, just kind of on the side, but horizontally where it was.

And it has a full access to the design, and I used a landing page for the designing academy, just kind of like the promotional page.

And it was a draft that I had on, you know, on hand, but the feedback was related to the copywriting.

It really wasn't about the composition, and it had 2 comments and one of them had repetitious text.

It's like this phrase is used over and over again.

And then that was obvious because it was placeholder information.

The other one, it talked about broken copy, and it says, students are eligible for a full refund within the 14 point of the course, and this is a phrase 14 point of is unclear.

And should likely be 14 days.

But it was none of that.

That was none of the text that was in there.

Somehow it garbled that up and took it as feedback.

It was about students are eligible for a full refund until the halfway point of the course.

And that, again, just draft content, but it was odd that it took that and it didn't, it didn't even see it.

So that's kind of like, that's not helpful.

It's just, it's AI slop, unfortunately.

The 3rd one I took a look at was this, uh, vibe coded, brutal feedback simulator.

So kind of funny in that it's what we're talking about.

And you upload a design and then it, it's kind of like a mad lib sort of thing.

It's kind of, it's funny, right?

It's meant to be a joke joke.

You choose who is giving you this feedback, the CEO, marketing manager sales lead.

And then you pick one so they had picked the CEO.

And it came back and it said, this looks like a cheap template from theme forest, not a $1000000 company.

The dated gradients on the funnel and bar charts scream early 2010s and completely undermine any sense of modern sophistication.

So yes.

We've heard things like that before.

So for maybe comic relief.

I don't know, maybe we hear enough of that in our daily refuse, we don't need a simulator to actually give us more of it.

But again, that sort of thing when you step back and say, well, what kind of feedback is that?

It's funny, and that's the point of it.

But if that was actual feedback, I'd want to dig into it and understand.

A cheap template from Theme Forest.

Well, what makes it cheap?

Uh, you know, it talks about dated gradients.

And we're talking about the visual appeal of it.

But I would go back up and ask about the function of it, and we may have to go back a step with this stakeholder and try to figure out what what the purpose is.

And when we're telling the purpose, we may have to go through and dig into it a bit more.

So all of this, this experiment, a little quick experiment on these different reviews, I don't think AI reviews, at least these 3 that you can't be trusted for a number of reasons.

And here's a question you can ask of this AI output or any other one that you're getting designed feedback.

If you gave your design to a designer, you most trust, and ask them for feedback, and they sent this back, what the system is sending back?

What would you think of them?

Would you be confused because you'd wonder if they even looked at the design?

Would you lose respect for them because it's inconsistent with the work you've seen them do?

You probably wouldn't ask them for feedback again, right?

This is like design designer to designer.

Like, we want trusted advisors.

And I think the same is with these AI reviews, but it goes a step further, right?

What's the attraction to using these AI reviews?

I think it's the same attraction as AI is in general, right?

It's fast and easy, accessible, right?

We have access to these things.

It's authoritative sounding responses.

It's definitive.

Do this.

Here's this chart.

Like it's very official.

And it's good enough, right?

The quotes good enough.

Right?

It's a thing.

But you got to parse through what's valuable and what's not.

So it's more investigation.

You got to kind of weed through.

What am I really going to ignore?

What's really valuable?

I don't really know.

There's so much volume of it.

I have to have to read the report and study it.

So I think there's a number of downsides of the AI reviews because you can't get a real explanation out of it, right?

It's it's a response.

It's generated.

It's a thing. But it's not a real explanation.

It's not like talking to a person.

And getting their insight, their thought, their history about it.

And you can't really have a actual conversation, not like a person.

It's not an understanding that sits behind the prompt.

Right?

It's an algorithm.

And it's prioritized for you, but perhaps it's not actually worth prioritizing.

Like the top issues.

And this one says the maximum of 10.

But are those really the top issues?

Maybe, maybe not.

But they look official.

So all of this, I don't think is really guiding you as a designer to learn why things work.

And that's an awesome thing.

When you're working with an actual designer and getting design reviews, You're you're learning, you get to get that, tap that experience of of a mentor or a peer or, you know, some other leader that's that's working through this and really picking it apart for you.

I don't think it really challenges you on your design decisions because that's the real value.

That's almost the super critical.

If we had to hone in on design reviews are the top skill. 2nd top skill, the deeper part of it is challenging your design decisions, because when you know why you've designed a thing, why you've put that there, you're going to be seeing that in other areas and able to apply it to other situations.

I know, it's like you're in the hot seat when you're getting your designs reviewed.

But it's really questions that show you where to dig deeper next time.

Right?

If you continually put borders around card elements and you don't have a reason, how can you explain it to a non-designer, why it's valuable to put it there?

Because there are valuable reasons to put it there.

And it all boils down to this fundamental problem with why.

Right?

The question why?

It's about purpose.

Why is always about purpose.

And I take it that step further of purpose being the intended outcome.

Purpose is really what drives people, all of us to excel, to do better, to build, to create, we're driven for this outcome that's out there.

We haven't gotten to it yet, but we want to get there and we want to make something that helps us get there.

And as designers, we love making things.

The AI doesn't know your features, specific outcomes in the context of your app, the strategy of your app, user goals, company goals.

All those contacts might get built in or you might be able to add those in.

But it doesn't know when to bend the rules, right?

It doesn't have a strategy.

It doesn't have an intuition.

It definitely doesn't have emotion.

And it's fascinating because we try so hard in these design reviews to take emotion out of it.

I always like to think of it as objective, right?

We're trying to get to an objective view.

Well, AI is so objective that there is no emotion available to it.

So we don't get the heart of it.

We're people, people.

We're 2 sides, rational and emotional, both and.

And it doesn't AI doesn't have that.

So it's going to be missing from these reports.

I think a human design review has purpose built into it, just by doing it.

And with AI, you'll never get a human response.

But think of it this way.

If you're building for people to use your software, your human users really require a human design review.

Because that's how we're going to relate.

So this is such a critical skill about how to get actionable feedback, not just feedback.

Even if you use AI for screen generation or experimentation, you still need to be able to look at it, categorize it, and map each of those design decisions that the AI made and map them back to a purpose.

And so it's a skill that you can learn with practice.

And to help you practice.

I put that free 30 minute quick course together in getting actionable feedback.

So you can go to academy.designey.com, and I've got the quick course there.

You can sign up, try it out.

Let me know if you have any questions.

But it is something that if you take that, I've got a downloadable there with you.

You can take it with your next meeting and you can walk through it.

Take another designer, try it out.

It's so exciting.

I hope that you'll use it.

But if you want to get involved in other ways, just continue to follow, subscribe to the daily sprint on this platform you're listening to.

If you have any questions, you did the exercise, you did the lists, you, I said something maybe here that you like have a question about, go to designy.com/ask.

And while you're there, sign up for the free newsletter at designy.com. That's “design” with a ”y” dot com.

So thanks for listening to The Daily Sprint today.

Remember, today is a great day to design with a why.

See you next time.