--- name: disaster-recovery description: Plan, implement, and test disaster recovery capabilities including business continuity planning, RTO/RPO definition, failover architecture, DR testing, crisis communication, and recovery execution. Use when creating DR plans, defining recovery objectives, designing failover systems, conducting DR drills, managing crisis communication, or executing disaster recovery. Triggers on phrases like "disaster recovery", "DR plan", "business continuity", "RTO", "RPO", "failover", "BCP", "DR test", "crisis management", "recovery architecture". --- # Disaster Recovery & Business Continuity Ensure organizational resilience through comprehensive disaster recovery planning, testing, and execution. ## Workflow ### 1. Business Impact Analysis & Planning 1. **Business impact analysis (BIA)**: - Critical business function identification - Dependency mapping (people, processes, technology, vendors) - Financial and operational impact quantification - Recovery time objective (RTO) definition per function - Recovery point objective (RPO) definition per function 2. **Risk assessment and scenario planning**: - Threat identification (natural, technical, human, cyber) - Vulnerability assessment - Likelihood and impact probability analysis - Risk mitigation strategy per scenario - Worst-case scenario planning 3. **DR plan development**: - DR strategy selection (hot, warm, cold site, cloud) - Recovery priority and sequence definition - Resource requirement identification - DR team role and responsibility assignment - Communication plan and contact list ### 2. DR Architecture & Infrastructure 1. **Data protection strategy**: - Backup strategy (full, incremental, differential) - Replication technology and frequency - Off-site and geo-redundant storage - Immutable and air-gapped backup for ransomware protection - Backup verification and integrity testing 2. **Failover infrastructure design**: - Primary and secondary site architecture - Cloud-based DR (AWS DR, Azure Site Recovery) - Network failover and DNS routing - Application failover sequence and dependency order - Capacity planning for DR site 3. **Technology stack resilience**: - High availability architecture (active-active, active-passive) - Load balancer and traffic routing failover - Database replication and failover - Identity and authentication redundancy - Security infrastructure continuity ### 3. DR Testing & Validation 1. **Testing strategy and cadence**: - Test type selection (documentation review, tabletop, simulation, parallel, full interruption) - Testing frequency by criticality (annual, bi-annual, quarterly) - Test scope and objective definition - Test environment preparation - Success criteria and acceptance threshold 2. **DR test execution**: - Test scenario and script development - Team briefing and role assignment - Test execution with time tracking - Issue and deviation logging - Test completion and recovery validation 3. **Post-test analysis and improvement**: - After-action review and lessons learned - RTO/RPO achievement assessment - Gap and deficiency identification - Improvement action plan development - DR plan update and version control ### 4. Incident Response & Recovery Execution 1. **Incident declaration and activation**: - Incident detection and assessment - DR activation criteria and authority - DR team notification and mobilization - Stakeholder communication activation - Escalation path execution 2. **Recovery execution**: - Recovery sequence execution per priority - System and application failover - Data restoration and validation - Service verification and testing - Operational status update 3. **Crisis communication**: - Internal communication (employees, management) - External communication (customers, partners, regulators) - Media and public communication - Status update cadence and channel - Post-incident communication and lessons learned ### 5. Recovery & Restoration 1. **Service restoration**: - Normal operations resumption criteria - Back-failover planning and execution - Data reconciliation and sync - Service validation and verification - Stakeholder notification of restoration 2. **Post-disaster review**: - Comprehensive incident review and analysis - DR plan effectiveness assessment - Financial impact assessment - Insurance claim preparation and filing - Regulatory notification and reporting 3. **Continuous improvement**: - DR plan update based on lessons learned - Infrastructure improvement implementation - Training and awareness program update - Vendor and partnership review - DR program maturity assessment ## Templates & Frameworks ### DR Plan Framework ``` DISASTER RECOVERY PLAN FRAMEWORK ================================= RECOVERY OBJECTIVES: Tier 1 (Mission Critical): RTO: < 1 hour RPO: < 15 minutes Systems: Core transaction processing, customer-facing applications Tier 2 (Business Critical): RTO: < 4 hours RPO: < 1 hour Systems: Internal applications, reporting, collaboration Tier 3 (Standard): RTO: < 24 hours RPO: < 4 hours Systems: Development, test, archival systems DR SITE CONFIGURATION: Primary site: [Location] — Full production DR site: [Cloud Region / Secondary Location] — Warm standby Failover mode: Semi-automated (manual activation, automated failover) DR capacity: 100% of production capacity Network connectivity: Dedicated fiber + satellite backup RECOVERY SEQUENCE: Phase 1 (0-30 min): Core infrastructure (network, identity, DNS) Phase 2 (30-60 min): Tier 1 applications and databases Phase 3 (1-4 hours): Tier 2 applications and services Phase 4 (4-24 hours): Tier 3 systems and full restoration DR TEAM ROSTER: DR Commander: [Name, role, phone, email, alternate] IT Lead: [Name, role, phone, email, alternate] Application Lead: [Name, role, phone, email, alternate] Data Lead: [Name, role, phone, email, alternate] Communications Lead: [Name, role, phone, email, alternate] Facilities Lead: [Name, role, phone, email, alternate] CONTACT DISTRIBUTION: Internal: Executive team, department heads, all employees External: Customers, partners, regulators, insurers Communication channels: SMS, email, phone tree, mass notification system Update frequency: Every 2 hours during active incident TESTING CADENCE: Documentation review: Quarterly Tabletop exercise: Semi-annually Simulation test: Annually Full failover test: Annually (off-peak) After-action review: Within 5 business days of each test ``` ### DR Test After-Action Report ``` DR TEST AFTER-ACTION REPORT ============================== TEST DETAILS: Test type: [Tabletop / Simulation / Parallel / Full Failover] Date: [Date] Duration: [Duration] Scenario: [Disaster scenario tested] Participants: [Names, roles] DR plan version: [Version] OBJECTIVE ASSESSMENT: Test objectives defined: [List] Objectives achieved: [X/X] Overall result: [Pass / Partial Pass / Fail] RTO/RPO ACHIEVEMENT: Tier 1 systems: Target RTO: 1 hour | Actual: [X minutes] | Result: [Pass/Fail] Target RPO: 15 min | Actual: [X minutes] | Result: [Pass/Fail] Tier 2 systems: Target RTO: 4 hours | Actual: [X hours] | Result: [Pass/Fail] Target RPO: 1 hour | Actual: [X minutes] | Result: [Pass/Fail] ISSUES IDENTIFIED: 1. [Issue description] Severity: [Critical / High / Medium / Low] Root cause: [Description] Impact: [Description] Resolution: [Immediate fix if applied] 2. [Issue description] Severity: [Critical / High / Medium / Low] Root cause: [Description] Impact: [Description] Resolution: [Immediate fix if applied] LESSONS LEARNED: What worked well: [List] What didn't work: [List] Unexpected challenges: [List] Plan gaps identified: [List] IMPROVEMENT ACTIONS: 1. [Action item] — Owner: [Name] — Due: [Date] — Priority: [High/Med/Low] 2. [Action item] — Owner: [Name] — Due: [Date] — Priority: [High/Med/Low] 3. [Action item] — Owner: [Name] — Due: [Date] — Priority: [High/Med/Low] RECOMMENDATIONS: DR plan updates needed: [List] Infrastructure improvements: [List] Training gaps identified: [List] Next test focus area: [Recommendation] ``` ## Integration Points - Cloud DR services (AWS Disaster Recovery, Azure Site Recovery, Zerto, VMware): DR infrastructure - Backup platforms (Veeam, Commvault, Rubrik, Veritas): Data protection - Mass notification systems (Everbridge, OnSolve, AlertMedia): Emergency communication - Business continuity management platforms (Resilient, Everbridge, Fusion): BCM orchestration - Monitoring platforms: System health and failover trigger - Communication platforms: Crisis communication - IT service management (ServiceNow): Incident and problem management - Cybersecurity platforms: Threat detection and response ## Edge Cases - **Ransomware during DR**: Immutable backup verification; air-gapped recovery point; clean state restoration; incident response coordination; forensic investigation - **Extended outage beyond planned RTO**: Extended operations at DR site; resource exhaustion planning; vendor dependency management; employee fatigue and shift management - **Multi-region/cloud disaster**: Cross-region replication strategy; DNS failover complexity; data consistency challenges; regulatory data sovereignty compliance - **Pandemic/business continuity**: Remote work activation; workforce health and safety; supply chain disruption; customer communication; long-term operational adjustment - **Regulated industry DR (financial, healthcare)**: Regulatory notification requirements; compliance documentation; audit trail maintenance; regulatory testing requirements ## Output ### DR Readiness Dashboard ``` DISASTER RECOVERY READINESS — April 2025 ========================================== OVERALL DR READINESS SCORE: 87/100 ✓ RECOVERY OBJECTIVE STATUS: Tier 1 (Mission Critical): RTO compliance: 98% ✓ (last test: 45 min, target: 60 min) RPO compliance: 100% ✓ (last test: 12 min, target: 15 min) Tier 2 (Business Critical): RTO compliance: 95% ✓ (last test: 3.2 hrs, target: 4 hrs) RPO compliance: 97% ✓ (last test: 52 min, target: 60 min) Tier 3 (Standard): RTO compliance: 100% ✓ RPO compliance: 100% ✓ DATA PROTECTION: Backup coverage: 99.2% ✓ Backup success rate: 98.7% ✓ Last full backup verification: 3 days ago ✓ Immutable backup status: Active ✓ Geo-redundant replication: Active (2 regions) RPO actual (avg): 18 minutes INFRASTRUCTURE READINESS: DR site capacity: 100% available Network failover tested: Active ✓ DNS failover configuration: Verified ✓ Identity redundancy: Active ✓ Security infrastructure: Redundant ✓ TESTING STATUS: Last tabletop exercise: 45 days ago ✓ Last simulation test: 120 days ago Last full failover test: 340 days ago (due within 30 days) Test pass rate (last 4 tests): 100% ✓ Open issues from last test: 3 (all medium, resolution in progress) PLAN GOVERNANCE: DR plan version: 4.2 (current) Last plan review: 30 days ago ✓ Contact list accuracy: Verified within 30 days ✓ Team training completion: 94% Awareness campaign: Active IMPROVEMENT TRACKING: Open improvement actions: 7 Completed this quarter: 12 Infrastructure upgrades planned: 3 Next test scheduled: [Date] ``` ## Trigger Phrases "disaster recovery", "DR plan", "business continuity", "RTO", "RPO", "failover", "BCP", "DR test", "crisis management", "recovery architecture", "backup strategy", "failover test", "incident response", "crisis communication", "after-action review"