86 lines
2.5 KiB
Markdown
86 lines
2.5 KiB
Markdown
# 20. Device Farm Testing
|
|
|
|
meta:
|
|
id: ios-production-20
|
|
feature: ios-production
|
|
priority: P2
|
|
depends_on: []
|
|
tags: [testing, device-farm, quality]
|
|
|
|
objective:
|
|
- Test the iOS app on a diverse range of physical devices using a device farm service
|
|
|
|
deliverables:
|
|
- Device farm test configuration
|
|
- Tests running on multiple iOS versions and devices
|
|
- Test results and reports
|
|
- Bug fixes from device-specific issues
|
|
|
|
steps:
|
|
1. Choose device farm service:
|
|
- AWS Device Farm
|
|
- Firebase Test Lab (limited iOS support)
|
|
- BrowserStack App Live
|
|
- Sauce Labs
|
|
- Or manual testing on physical devices
|
|
2. Configure test suite:
|
|
- Upload IPA build
|
|
- Configure XCTest UI test bundle
|
|
- Select device matrix:
|
|
- iPhone SE (3rd gen) — iOS 17
|
|
- iPhone 12 — iOS 16
|
|
- iPhone 14 Pro — iOS 17
|
|
- iPhone 15 Pro Max — iOS 18
|
|
- iPad Pro 12.9" — iOS 18
|
|
- iPad mini — iOS 17
|
|
3. Define test scenarios:
|
|
- Install and launch
|
|
- Complete onboarding
|
|
- Login and dashboard navigation
|
|
- Run security scan
|
|
- View alerts and details
|
|
- Settings navigation
|
|
- Background/foreground transitions
|
|
4. Run tests and collect results:
|
|
- Execute automated test suite
|
|
- Collect screenshots and videos
|
|
- Gather performance metrics
|
|
- Document device-specific issues
|
|
5. Fix device-specific bugs:
|
|
- Notch/safe area issues on different models
|
|
- Dynamic Island interactions
|
|
- iPad multitasking support
|
|
- Performance on older devices
|
|
6. Add device farm to CI:
|
|
- Trigger tests on release builds
|
|
- Block release on critical failures
|
|
- Archive results for compliance
|
|
|
|
tests:
|
|
- Device: All selected devices pass core tests
|
|
- Regression: No new failures compared to previous run
|
|
- Performance: App responsive on all devices
|
|
|
|
acceptance_criteria:
|
|
- Tests run on 6+ different iOS devices
|
|
- Tests cover iOS 16, 17, and 18
|
|
- All critical flows pass on all devices
|
|
- Screenshots and videos from all test runs
|
|
- Device-specific issues identified and fixed
|
|
- Device farm integrated into release CI
|
|
- Test results archived for 1 year
|
|
- No crashes on any tested device
|
|
- Performance acceptable on oldest supported device
|
|
|
|
validation:
|
|
- Run device farm tests → all devices pass
|
|
- Review screenshots → UI looks correct on all sizes
|
|
- Check videos → no stuttering or crashes
|
|
- Compare with previous run → no regressions
|
|
|
|
notes:
|
|
- AWS Device Farm is most comprehensive for iOS
|
|
- Physical device testing is ideal but expensive
|
|
- Focus on edge cases: small screens, old iOS versions
|
|
- Test cellular vs WiFi behavior if possible
|