# 20. Device Farm Testing

meta:
  id: ios-production-20
  feature: ios-production
  priority: P2
  depends_on: []
  tags: [testing, device-farm, quality]

objective:
- Test the iOS app on a diverse range of physical devices using a device farm service

deliverables:
- Device farm test configuration
- Tests running on multiple iOS versions and devices
- Test results and reports
- Bug fixes from device-specific issues

steps:
1. Choose device farm service:
   - AWS Device Farm
   - Firebase Test Lab (limited iOS support)
   - BrowserStack App Live
   - Sauce Labs
   - Or manual testing on physical devices
2. Configure test suite:
   - Upload IPA build
   - Configure XCTest UI test bundle
   - Select device matrix:
     - iPhone SE (3rd gen) — iOS 17
     - iPhone 12 — iOS 16
     - iPhone 14 Pro — iOS 17
     - iPhone 15 Pro Max — iOS 18
     - iPad Pro 12.9" — iOS 18
     - iPad mini — iOS 17
3. Define test scenarios:
   - Install and launch
   - Complete onboarding
   - Login and dashboard navigation
   - Run security scan
   - View alerts and details
   - Settings navigation
   - Background/foreground transitions
4. Run tests and collect results:
   - Execute automated test suite
   - Collect screenshots and videos
   - Gather performance metrics
   - Document device-specific issues
5. Fix device-specific bugs:
   - Notch/safe area issues on different models
   - Dynamic Island interactions
   - iPad multitasking support
   - Performance on older devices
6. Add device farm to CI:
   - Trigger tests on release builds
   - Block release on critical failures
   - Archive results for compliance

tests:
- Device: All selected devices pass core tests
- Regression: No new failures compared to previous run
- Performance: App responsive on all devices

acceptance_criteria:
- Tests run on 6+ different iOS devices
- Tests cover iOS 16, 17, and 18
- All critical flows pass on all devices
- Screenshots and videos from all test runs
- Device-specific issues identified and fixed
- Device farm integrated into release CI
- Test results archived for 1 year
- No crashes on any tested device
- Performance acceptable on oldest supported device

validation:
- Run device farm tests → all devices pass
- Review screenshots → UI looks correct on all sizes
- Check videos → no stuttering or crashes
- Compare with previous run → no regressions

notes:
- AWS Device Farm is most comprehensive for iOS
- Physical device testing is ideal but expensive
- Focus on edge cases: small screens, old iOS versions
- Test cellular vs WiFi behavior if possible