You test landing pages. You test email subject lines. You test button colors. Why aren't you testing job posts?
A single change—like adding salary or tweaking the title—can increase applications by 30-50%. But most companies never test. They post once, hope for the best, and wonder why they got 3 applications.
This guide shows you exactly what to test, how to run valid experiments, and what results to expect based on real data.
What Elements to Test
Not all job post elements are worth testing. Focus on the high-impact variables that actually move the needle.
High-Impact Tests (Start Here)
1. Job Title
HIGHEST IMPACTWhy it matters:
Title determines if anyone clicks your job post. It's the only element visible in search results.
What to test:
- • Standard vs creative titles ("Software Engineer" vs "Code Wizard")
- • Seniority level clarity ("Engineer" vs "Senior Engineer" vs "Engineer II")
- • Remote/location in title vs separate field
- • Tech stack in title ("React Engineer" vs "Frontend Engineer")
Example Test:
A: "Marketing Manager"
B: "Growth Marketing Manager (Remote)"
Hypothesis: Adding "Growth" and "Remote" increases click-through rate
2. Salary Transparency
PROVEN WINNERWhy it matters:
91% of candidates want to see salary. But does showing it help or hurt application quality?
What to test:
- • Salary range vs no salary
- • Wide range ($60-90K) vs narrow range ($70-80K)
- • Starting minimum ("From $70K") vs range
- • Base salary vs total comp (base + equity + bonus)
Example Test:
A: No salary mentioned
B: "$80,000 - $100,000 based on experience"
Measure: Application volume AND quality (screening pass rate)
3. Requirements Count
QUICK WINWhy it matters:
Long requirement lists scare away qualified candidates (especially women and underrepresented groups).
What to test:
- • 5 requirements vs 15 requirements
- • "Must-haves" only vs "Must-haves + Nice-to-haves"
- • Years of experience (3+ years) vs skill-based (proficient in X)
- • Degree required vs degree OR equivalent experience
Example Test:
A: 12 bullet point requirements
B: 5 must-haves + 3 nice-to-haves (clearly labeled)
Hypothesis: Shorter list increases applications without hurting quality
4. Call-to-Action
OFTEN IGNOREDWhy it matters:
The CTA determines if interested candidates actually apply or abandon.
What to test:
- • Button text ("Apply Now" vs "Join Our Team" vs "Get Started")
- • Application friction (full form vs "Apply with LinkedIn")
- • Timeline transparency ("Hear back in 5 days" vs no timeline)
- • Required fields (just resume vs resume + cover letter + portfolio)
Example Test:
A: Standard "Apply Now" button, 8 required fields
B: "Apply in 2 Minutes" button, 3 required fields
Measure: Application completion rate (started vs finished)
Medium-Impact Tests (Test After Basics)
-
•
Opening paragraph tone: Formal vs casual, "we need" vs "you'll get" framing
-
•
Company description length: 2 sentences vs 2 paragraphs
-
•
Benefits placement: Top of post vs bottom of post
-
•
Formatting: Bullet points vs paragraphs, emoji use vs no emoji
-
•
Post length: 300 words vs 600 words vs 1000 words
increase in applications when salary range was added to job posts
Internal data from 200+ EasyApply customers, 2024
Test Job Posts Without Extra Tools
EasyApply lets you run A/B tests on job titles, descriptions, and CTAs. Track performance automatically.
Start Testing Free →How to Run Tests
A/B testing isn't hard, but there's a right way and a wrong way. Here's the playbook.
The A/B Testing Process
Form a Hypothesis
Don't test randomly. Have a reason.
Example: "Including salary range will increase application rate by 20-30% because candidates filter by salary and appreciate transparency."
Create Two Versions
Change ONE variable only. Otherwise you won't know what caused the difference.
- ✓ Good: Version A has no salary, Version B has salary range (1 change)
- ✗ Bad: Version B has salary + different title + shorter requirements (3 changes)
Split Traffic Evenly
50% of viewers see Version A, 50% see Version B. Random assignment.
How to split:
- • Same job board, alternate versions daily (Mon/Wed/Fri = A, Tue/Thu/Sat = B)
- • Post on two job boards simultaneously (Indeed = A, LinkedIn = B) *only if boards have similar audiences
- • Use platform with built-in A/B testing (EasyApply, some ATS tools)
Define Success Metrics
What are you measuring?
- Primary metric: Application rate (applications / views)
- Secondary metrics: Quality (% passing screening), time-to-apply, completion rate
- Warning: Don't just measure volume. 100 bad applicants < 10 qualified ones.
Let It Run (Patience Required)
Don't stop early. You need statistical significance (more on this below).
Typical test duration: 7-14 days or until 100+ views per version
Analyze & Implement Winner
If there's a clear winner (and it's statistically significant), use that version going forward.
Then test something else. Continuous improvement.
Common Testing Mistakes
-
✗
Testing too many variables at once
You won't know what caused the change
-
✗
Stopping the test too early
Random variance can look like a trend. Wait for significance.
-
✗
Unequal traffic split
Version A gets 80 views, Version B gets 20 views = invalid test
-
✗
Testing on different platforms/audiences
LinkedIn vs Indeed have different users. Not comparable.
-
✗
Ignoring quality metrics
More applications ≠ better if they're all unqualified
Sample Sizes & Stats
"After 3 applications, Version B is winning!" Not how this works. You need enough data to trust the results.
Minimum Sample Size
Rule of thumb for job post testing:
Absolute minimum: 100 views per version
Below this, results are too noisy to trust
Better: 250+ views per version
Gives you ~90% confidence in results
Ideal: 500+ views per version
Strong statistical significance, reliable insights
Statistical Significance Explained (Simply)
What it means:
Statistical significance tells you if the difference between A and B is real or just random luck.
The threshold:
In A/B testing, we typically want 95% confidence (p-value < 0.05). This means there's less than a 5% chance the result is random.
Example:
Version A: 200 views → 20 applications (10% rate)
Version B: 200 views → 30 applications (15% rate)
Result: Statistically significant at 95% confidence
Version B is likely 50% better. Trust this result.
How Long to Run Tests
| Job Post Traffic | Typical Test Duration | Views Needed |
|---|---|---|
| High traffic (50+ views/day) | 4-7 days | 200-350 views total |
| Medium traffic (20-50 views/day) | 7-14 days | 280-700 views total |
| Low traffic (<20 views/day) | 14-30 days | 280-600 views total |
Too few views to test?
If your job posts get <10 views total, A/B testing won't work. Focus on increasing visibility first (promote on paid channels, better SEO, more job boards).
confidence level needed before trusting A/B test results
Don't call a winner too early or you'll make the wrong decision
Calculate Statistical Significance Automatically
EasyApply's built-in A/B testing shows you when results are significant. No manual math required.
Try A/B Testing Free →Tools for Testing
You don't need fancy enterprise tools. Here are practical options for small businesses.
Testing Tools Comparison
Manual Testing (Free)
DIYPost two versions on the same platform at different times. Track results in a spreadsheet.
Pros
- • Free
- • Works anywhere
- • Simple
Cons
- • Manual tracking
- • Time-sensitive bias
- • No auto-stats
Job Board Native Tools
PLATFORM-SPECIFICSome job boards (LinkedIn, Indeed Premium) offer built-in A/B testing for titles.
Pros
- • Integrated with platform
- • Auto traffic split
- • Easy to use
Cons
- • Limited to one platform
- • Often paid feature
- • Can't test full post
ATS with A/B Testing (EasyApply, Greenhouse, Lever)
RECOMMENDEDModern ATS tools let you test job posts across multiple boards from one interface.
Pros
- • Test across all boards
- • Auto significance calc
- • Quality metrics tracked
- • Historical data
Cons
- • Requires ATS subscription
- • Learning curve
Google Sheets + Stats Calculator
FREE + POWERFULTrack data manually, use free online calculators for significance testing.
Tools you'll need:
- • Google Sheets (tracking)
- • AB Test Calculator (abtestguide.com/calc)
- • Job board analytics (for view counts)
Our recommendation for small businesses:
Start with manual testing in a spreadsheet. If you're posting 5+ jobs/year, invest in an ATS with built-in A/B testing (like EasyApply).
Real Case Studies
Here's what actually happened when companies tested job posts. Real numbers, real results.
Case Study 1: Adding Salary Range
Test:
- Version A: "Competitive salary and equity"
- Version B: "$90,000 - $120,000 + equity (0.1-0.25%)"
Role:
Senior Product Manager
Results (14 days, 450 total views):
Version A (No Salary)
- • 220 views
- • 18 applications (8.2% rate)
- • 6 qualified (33% quality)
Version B (With Salary)
- • 230 views
- • 32 applications (13.9% rate)
- • 14 qualified (44% quality)
Outcome:
+70% more applications, +33% better quality. Statistically significant. Winner: Version B.
They now include salary in all job posts.
Case Study 2: Shortening Requirements
Test:
- Version A: 14 bullet point requirements
- Version B: 5 must-haves + 3 nice-to-haves (clearly separated)
Role:
Marketing Coordinator
Results (10 days, 380 total views):
Version A (14 Requirements)
- • 190 views
- • 12 applications (6.3% rate)
- • 8 qualified (67% quality)
Version B (8 Requirements)
- • 190 views
- • 26 applications (13.7% rate)
- • 16 qualified (62% quality)
Outcome:
+117% more applications, quality stayed high. Statistically significant. Winner: Version B.
Long requirement lists were scaring away qualified candidates.
Case Study 3: Job Title Specificity
Test:
- Version A: "Full Stack Engineer"
- Version B: "Full Stack Engineer (React/Node)"
Results (7 days, 520 total views):
Version A (Generic Title)
- • 250 views
- • 35 applications (14% rate)
- • 12 qualified (34% quality)
Version B (Tech Stack in Title)
- • 270 views
- • 28 applications (10.4% rate)
- • 18 qualified (64% quality)
Outcome:
Fewer applications but +88% better quality. Statistically significant. Winner: Version B.
Specific tech stack filtered out mismatched candidates early, saved screening time.
Key Takeaways from Case Studies
-
→
Salary transparency wins. More applications + better quality in every test we've seen.
-
→
Shorter requirements = more applicants. Long lists scare people away.
-
→
More applicants ≠ always better. Sometimes fewer, higher-quality applicants is the win.
-
→
Test what matters for YOUR role. Engineer tests won't apply to sales roles.
Start Testing Your Job Posts Today
EasyApply makes A/B testing effortless. Test titles, descriptions, salary display, and more. See what actually works for your roles.
No credit card required. Run unlimited A/B tests.
A/B Testing Checklist
Continue Reading
Job Posting Optimization
The complete guide to better job posts.
Best Job Boards 2025
Where to post for maximum ROI.
Employer Branding in Job Posts
Stand out to candidates.
Job Title Best Practices
Test these first.
Include Salary in Posts?
The data says yes.
Why No Applicants?
Test to fix these issues.