Kordant/tasks/ios-production/20-device-farm-testing.md

20. Device Farm Testing

meta: id: ios-production-20 feature: ios-production priority: P2 depends_on: [] tags: [testing, device-farm, quality]

objective:

• Test the iOS app on a diverse range of physical devices using a device farm service

deliverables:

• Device farm test configuration
• Tests running on multiple iOS versions and devices
• Test results and reports
• Bug fixes from device-specific issues

steps:

1. Choose device farm service:
   • AWS Device Farm
   • Firebase Test Lab (limited iOS support)
   • BrowserStack App Live
   • Sauce Labs
   • Or manual testing on physical devices
2. Configure test suite:
   • Upload IPA build
   • Configure XCTest UI test bundle
   • Select device matrix:
     • iPhone SE (3rd gen) — iOS 17
     • iPhone 12 — iOS 16
     • iPhone 14 Pro — iOS 17
     • iPhone 15 Pro Max — iOS 18
     • iPad Pro 12.9" — iOS 18
     • iPad mini — iOS 17
3. Define test scenarios:
   • Install and launch
   • Complete onboarding
   • Login and dashboard navigation
   • Run security scan
   • View alerts and details
   • Settings navigation
   • Background/foreground transitions
4. Run tests and collect results:
   • Execute automated test suite
   • Collect screenshots and videos
   • Gather performance metrics
   • Document device-specific issues
5. Fix device-specific bugs:
   • Notch/safe area issues on different models
   • Dynamic Island interactions
   • iPad multitasking support
   • Performance on older devices
6. Add device farm to CI:
   • Trigger tests on release builds
   • Block release on critical failures
   • Archive results for compliance

tests:

• Device: All selected devices pass core tests
• Regression: No new failures compared to previous run
• Performance: App responsive on all devices

acceptance_criteria:

• Tests run on 6+ different iOS devices
• Tests cover iOS 16, 17, and 18
• All critical flows pass on all devices
• Screenshots and videos from all test runs
• Device-specific issues identified and fixed
• Device farm integrated into release CI
• Test results archived for 1 year
• No crashes on any tested device
• Performance acceptable on oldest supported device

validation:

• Run device farm tests → all devices pass
• Review screenshots → UI looks correct on all sizes
• Check videos → no stuttering or crashes
• Compare with previous run → no regressions

notes:

• AWS Device Farm is most comprehensive for iOS
• Physical device testing is ideal but expensive
• Focus on edge cases: small screens, old iOS versions
• Test cellular vs WiFi behavior if possible