How rewriting one email changed a product's business model
Content Strategy · Conversation Design · 2019 · 4 min read
In 2019, I was the conversation designer for Penny, Credit Karma's financial coaching chatbot. Most of my work was inside the app — voice, tone, and branching logic for conversations about credit health, loans, insurance, debt. This story isn't about one of those. It's about an email that failed, and what happened when I rewrote it.
Credit Karma had a data blind spot. When a member applied for a credit product through the app — a card, a loan — and got handed off to the partner, we lost visibility. We could sometimes see a hard inquiry land on their credit report. Occasionally a new tradeline would show up weeks later. But we had no reliable way to connect those signals back to a specific product recommendation, which meant no way to know whether our recommendations were actually working.
There was an internal debate about why people converted at all. One popular theory treated it like the candy aisle at checkout — members came to check their score, saw an offer they could plausibly get, and bought on impulse. Anish, Director of Core Product, wanted real data to settle it.
The only way to get that data was to ask users directly.
The plan: trigger an email when a member had (1) followed a partner offer out of the app and (2) picked up a hard inquiry from that same partner. The email would deep-link into a Penny conversation asking what happened.
We had no marketing support and were moving fast. Someone spun up the trigger email fast. Generic subject line, generic ask. It read like every other feedback request landing in someone's inbox that week: "Give us feedback on your recent experience." Body copy: "We saw you recently applied for a product... tell us how we did."
We sent it to 10,000 users as a test. The numbers came back:
Not underwhelming. Broken. At that response rate, getting to statistical significance was going to take forever, if it ever got there.
I asked to see the actual email. The offer wasn't the problem, and neither was the targeting or the Penny conversation behind it. The subject line was talking about what we wanted to know. It should have been talking about what the user already cared about.
Here's what we knew about Credit Karma members: they are hyper-alert to unexpected activity on their credit. A hard inquiry they don't immediately recognize registers as a threat. Not a survey request. We were sitting on exactly that signal — every recipient of this email had a fresh hard inquiry on file — and burying it under generic feedback-request language.
New subject line: "There's a new hard inquiry to your credit, was this you?"
The body led with Penny, positioned as looking out for the member's financial safety. The CTA was "Review Activity" — the same thing a worried person would already want to do.
After the tap-through, Penny showed the member which organization had made the inquiry and asked, simply: was this you? If yes, the conversation asked whether they'd been approved for the product: congratulations if so, a path to a better-fit product if not. If they didn't recognize the inquiry, Penny routed them straight into fraud-reporting guidance.
Same targeting logic. Same underlying data. Completely different framing.
Every number moved by an order of magnitude in the right direction. The more interesting result showed up downstream. Members who went through the Inquiry Assistant conversation came back to Credit Karma roughly three times in rapid succession before they ever purchased a financial product — a trust pattern we'd never been able to see before.
That data did more than settle Anish's debate. It shifted how leadership thought about partner placements. Less like an ad slot. More like curation — Penny surfacing the right offer because it understood the member's situation, not because a partner paid for it.
Nothing about the underlying mechanics changed between the two versions of this email — same trigger conditions, same data, same destination conversation. The only thing that changed was which side of the message I wrote from: what the company wanted to know, versus what the person opening the email was already worried about. That's most of what content strategy actually is. The system was already built. The email just had to stop talking to itself.