The internal communicator’s guide to A/B testing

You’re staring at a 28% open rate on last week’s all-company email…the one you spent two hours writing. The one with the SUPER important benefits update that everyone actually needed to read.

So you Google “how to improve email open rates” and every result is about marketing campaigns, conversion funnels, and subscriber lists. You try “A/B testing emails” — same thing. Guides built for people optimizing e-commerce newsletters, not people trying to get 4,000 employees to read a note from the CEO.

OK so here’s the good news: A/B testing works beautifully for internal comms. And it’s way simpler than those marketing guides make it sound. (Speaking as a marketer, haha.)

You don’t need a statistics background, you don’t need fancy software, and you definitely don’t need anyone’s permission to start!

You can start with just two subject lines and a willingness to pay attention to what happens next. 🙂

 

Why it looks different on the inside

Most A/B testing advice assumes you have a massive audience, a revenue goal, and zero personal relationship with the people you’re emailing. Internal comms flips all of that.

Your audience is smaller, and that’s such a cool advantage. A marketing team might A/B test with 50,000 subscribers and publish results in a case study. You’re working with 2,000 or maybe 8,000 employees. Your results are directional signals, not publishable research — and directional signals are exactly what you need. You’re not defending a thesis, you’re probably just trying to figure out whether “Q2 benefits update” or “Your benefits are changing in July” gets more people to actually open the email!

You have a relationship with your audience. Marketers are earning attention from strangers. You’re communicating with people who already have a reason to care about what you’re saying — they work at your company. That’s a fundamentally different starting point. Testing is NOT about tricking people into clicking. It’s about being respectful enough to learn what resonates with your colleagues instead of just… guessing.

The politics can feel a little funny, but don’t overthink it. The first time you mention “experimenting” with internal emails, it might feel a little funny, so don’t call it experimenting on employees. Call it what it is: you’re testing subject lines to make sure important information actually reaches people. You’re testing whether Tuesday morning or Thursday afternoon is a better send time — because you’d rather know than assume. Nobody objects to that!

 

What to test first (and what to happily ignore)

Start with subject lines. Always subject lines.

They’re the single biggest factor in whether your email gets opened, they’re easy to test, and they don’t require you to create two entirely different versions of anything. You write two subject lines instead of one. Done!

A few IC-specific pairs worth trying:

• Informational vs. personal: “Company holiday schedule for 2026” vs. “Your 2026 holiday dates are here”
• Formal vs. conversational: “An update from the leadership team” vs. “What we decided at the offsite last week”
• Vague vs. specific: “Important HR announcement” vs. “We’re adding 4 new mental health benefits”

After you’ve run five or so subject line tests, move on to send times. Does your all-company email perform better at 9am Tuesday or 2pm Thursday? You probably have a hunch — test it. 🙂

After send times, try format. Does a shorter email with a link to a full article outperform a longer email with everything inline? Great test for month three.

Everything else? Ignore it until you’ve done at least five subject line tests. Button colors, header images, font sizes — those are marketing optimization rabbit holes that will eat your time without teaching you much about your employees.

A quick word of advice: with A/B tests, you typically want to test ONE small change or a TON of changes. This sounds confusing, but the gist of it is that you want to really know what performed better or worse. So you want to know if the subject line truly tested better, or if the entire design of your email made the click-through rate improve. Just make sure you know what the end result might tell you.

How to actually run your first test

This is less complicated than it sounds, I promise. You can do this with most email tools, and if yours doesn’t support A/B testing natively, you can do it manually with a list split.

Pick your email. Choose an upcoming all-company email — something that goes to your full employee list. Don’t start with a niche email that only goes to 200 people. You want enough volume that the results actually mean something.

Write two subject lines. Not ten, not five — just two! Make them genuinely different from each other, not just tiny little word swaps. “Q3 earnings recap” and “Q3 earnings summary” won’t teach you anything. “Q3 earnings recap” and “Here’s how we did last quarter” — now you’re learning something.

Split your list. Divide your employee list roughly in half. Most email platforms can do this automatically. If you’re doing it manually, an alphabetical split works — A through M gets subject line A, N through Z gets subject line B. It doesn’t need to be perfectly random. It needs to be roughly even.

Send and wait 24 hours. Both versions should go out the same time on the same day. Then — and this is the hard part — don’t look at the results for a full 24 hours. Internal email has a longer tail than marketing email. People check at different times depending on their schedules, time zones, and meeting loads. (Monday at 9am is a war zone in most inboxes. Give it time!)

Record the result. Write down both subject lines, both open rates, and the date. A simple spreadsheet with five columns — date, subject line A, open rate A, subject line B, open rate B — is all you need.

One test by itself doesn’t prove much. But you’re not going for proof from a single test — you’re building a body of evidence.

 

Reading your results (no stats degree required)

The simplest framework for deciding if a result matters: ask yourself, “Would I change my behavior based on this?”

If subject line A got a 52% open rate and subject line B got a 54% — nah. That’s noise. Either one was fine.

If subject line A got a 41% and subject line B got a 58%? OK YES. Something about the way you framed that email made a real difference, and you should pay attention to what it was.

You don’t need to calculate statistical significance. You need to notice patterns.

What matters most is accumulation. One test tells you almost nothing. Five tests start to reveal patterns — maybe specific, personal subject lines consistently outperform formal ones at your company. Ten tests give you real knowledge about your workforce’s communication preferences. That’s knowledge you literally cannot get any other way.

A quick note on benchmarks: Internal comms open rates are wildly different from marketing open rates. A marketing team celebrating a 25% open rate would be having a pretty good day. Internal comms benchmarks tend to run much higher — your all-company emails should be doing significantly better than that. If you’re not sure what “good” looks like for your company, that’s exactly why you start testing: your first few results become your baseline, and then you improve from there.

 

Talking about testing with leadership

Once you’ve got a few results under your belt, you’ll want to share what you’re learning. Maybe your VP asks how comms are performing, or maybe you want to advocate for better tools.

Try something like this: lead with measurement, not experimentation. “I’ve been measuring which types of subject lines get the highest open rates” feels very different than “I’ve been running experiments on our employee emails.” Same activity. Very different energy. Most leaders love hearing that their comms team is being rigorous about measurement.

Share your first win early. The first time a test produces a clear winner — and it will — tell someone about it. “I tested two subject lines on the benefits email and the specific one outperformed the vague one by 15 percentage points.” That takes thirty seconds to share and it builds credibility for everything that comes after.

Also, be sure to connect it to business outcomes when you can! Open rates by themselves are interesting. Open rates connected to business goals are persuasive. If your benefits enrollment email gets a 20% higher open rate because you tested the subject line, and that contributes to higher enrollment completion? That’s a story worth that’s absolutely worth telling at the leadership level.

 

Your next three months

Forget the twelve-month plan. You need a starting point and enough momentum to build a habit.

Month 1 is just two subject line tests on all-company emails. Record results in your spreadsheet. That’s it — don’t overcomplicate this!

Month 2, let’s try a send time test. Try sending the same type of email at two different times during two consecutive weeks. Note what you find!

Month 3, test a format change — try a shorter version of your regular newsletter against the full-length version. At the end of the month, pull together a one-page summary of what you’ve learned from your five or six tests and share it with your manager or leadership team. It doesn’t need to be a deck. A few bullet points in an email will do. 🙂

By the end of three months, you’ll know more about how your employees engage with internal email than most comms teams learn in a year. And you’ll have the receipts to prove it at your next strategy review!

 

Go test something this week

Let’s come back to that 28% open rate. It felt discouraging at the time, but now it’s something way more useful — a baseline. A number you can actually improve, with evidence, starting this week!

Your next all-company email is a chance to run your first test. Pick that email, write two genuinely different subject lines, split your list, and see what happens. It’s like fifteen extra minutes of work, tops!

You’re already doing the hard part — writing comms that matter. The next step is just making sure the right people actually see them.

And if you’re looking for a platform that makes this even easier — Workshop has A/B testing built right in to our internal communications platform, so you can skip the manual list splitting and truly test your outcomes side-by-side.

But start with the manual version first. The learning matters more than the tool, and then you’ll establish a really great argument for why an automated tool might make all the difference. 🙂

 

Need help with some ideas for A/B testing your internal comms?

Just ask Cici (our AI agent) to help spin up some ideas for you!

If you are not yet a Workshop customer, you can use a lightweight version of Cici here and get some ideas for subject lines, specific approaches, etc.

And if you are a Workshop customer, even better: just log in to the app and Cici will have full context of your internal communications and campaigns, and can recommend a much more custom, tailored approach! Just look for the little purple sun in the bottom right to get started. 🙂