Resilience is about more than just the technology: Reflections from the inaugural AWS Resilience Day, UK North
.webp)
Resilience is something every organisation knows it needs. But knowing it matters and being genuinely prepared when something goes wrong are two very different things.
That was one of the key messages from the inaugural AWS Resilience Day, which brought together members of the AWS UK tech community to explore how organisations can build, test and maintain resilience in an increasingly complex technology landscape.
Resilience is not just about building systems that never fail, it is about understanding what matters most to your business, preparing for disruption and giving your people the skills, processes and confidence to respond when the unexpected happens.
For organisations delivering critical services, the consequences of getting this wrong extend far beyond technology. They can affect customers, colleagues, business operations and, in some cases, essential services that people rely on every day.
Here are some of the key lessons we took away from the day.
One of the opening discussions, led by David Elliott, Director, Head of Technology at AWS, explored why resilience deserves greater attention at every level of an organisation.
He discussed a key challenge being that we tend to plan for the things we expect to happen. We consider familiar risks, assess the likelihood of disruption and put measures in place to manage them. But what about the events we haven't anticipated?
Rare, unexpected events – or ‘black swan’ events – can expose weaknesses that routine planning overlooks. These are precisely the moments when an organisation's resilience is put to the test.
Making the case for investment in resilience can be difficult, particularly when the benefits are not always immediately visible. However, the potential impact of a major disruption makes it a business-critical consideration, not simply a technical one.
This means starting with the business itself. What services are most important? Which customer journeys must be protected? How much disruption can the organisation tolerate, and what would the consequences be if a critical service became unavailable?
These questions need input from across the business, including technical teams, operational leaders and senior stakeholders. Without a shared understanding of priorities, it is difficult to make informed decisions about where to invest time and resources.
Chris Strug, a Senior Technical Account Manager from AWS reinforced an important principle – the goal isn't to create systems that never fail. It's to manage disruption effectively when failures inevitably occur.
Resilience responses involve three connected activities:
This is a useful way of thinking about resilience because it moves the conversation beyond prevention alone. Even with robust architecture and the appropriate safeguards, failures can still happen. What matters is how well an organisation anticipates, contains and recovers from them.
The same principle applies to disaster recovery and business continuity. Plans that were created several years ago may no longer reflect today's systems, dependencies, operating models or customer expectations. They need to evolve alongside the business.
Resilience should therefore be treated as a continuous lifecycle whereby objectives are established, appropriate measures are designed and implemented, they are then tested and evaluated, and organisations learn from the results before iterating. Crucially, testing and validation are essential. A plan that looks convincing on paper is not necessarily a plan that will work under pressure.
A further takeaway from the day was the importance of being realistic about resilience objectives.
Not every application, system or business process needs the same level of protection. Trying to make everything equally resilient can introduce unnecessary cost, complexity and operational overhead.
Instead, organisations should identify their critical business activities and the customer journeys that depend on them. Working with the right stakeholders, they can then assess what failure would mean, where the greatest risks sit and what level of disruption is acceptable.
This is where clear recovery objectives and service level objectives can help. They provide a practical way to define expectations, measure performance and understand whether resilience measures are delivering the intended outcomes.
The challenge is to balance business requirements with the realities of cost, complexity and operational effort. The right approach will look different for every organisation. The important thing is to make those decisions deliberately, based on business needs rather than technology alone.
One of the most valuable themes from the day was the importance of the human side of resilience. This was particularly relevant to the session delivered by our Leighton colleague, Mark Sailes.
Technology, architecture and documented processes all play an important role, but they are only part of the picture. When a serious incident occurs, people need to interpret incomplete information, make decisions quickly, communicate clearly and work together to resolve the problem.
These are not skills that can be developed solely by writing a disaster recovery plan or investing in more sophisticated technology, they need to be practiced.
You can read more about Mark’s talk here.
Under normal circumstances, teams have time to investigate, discuss options and make considered decisions. During a major incident, that luxury can disappear.
Pressure, uncertainty and competing priorities can make communication more difficult. People may focus on their own area of responsibility rather than the wider impact, or hesitate because they are unsure who should make a decision. Even experienced technical teams can find it challenging to maintain perspective when the situation is changing rapidly.
This is why the softer skills are so important. Clear communication helps teams share information and maintain a common understanding of what is happening. Collaboration helps people draw on different expertise rather than working in isolation. Leadership provides direction when there is uncertainty and good decision-making helps teams distinguish between what needs immediate action and what can wait.
Equally important is the ability to remain calm, adapt as new information emerges and recognise when an initial assumption may have been wrong.
These capabilities are not separate from technical resilience. They are what enable people to put technical capabilities and established processes into practice when they are needed most.
If we expect teams to perform well under pressure, we need to give them opportunities to practise before the pressure arrives. That means moving beyond checking whether a system can recover and testing how people respond when it does not behave as expected.
Scenario-based exercises and incident simulations can help teams practice handling uncertainty, communicating across different functions and making decisions with incomplete information. They can also reveal gaps in responsibilities, escalation routes and decision-making authority that might otherwise only become apparent during a real incident.
Exercises don't always need to be highly technical. A facilitated discussion around a realistic scenario can be just as valuable for exploring how teams share information, involve the right stakeholders and agree on priorities.
The aim is not to catch people out or expect every decision to be perfect. It is to create a safe environment in which teams can learn, build confidence and understand how they work together.
Afterwards, the debrief matters just as much as the exercise itself. What went well? Where did communication break down? Were decisions clear? Did people have the information and authority they needed? What would they do differently next time?
Those lessons should feed back into training, operational processes and future exercises.
Another important lesson is that resilience cannot sit with a single individual or technical team. A major incident may involve application teams, infrastructure specialists, operational colleagues, business leaders, suppliers and external partners. Each brings a different perspective, and effective recovery depends on those people working together.
Building relationships before an incident makes it easier to collaborate when one occurs. Teams that understand one another's responsibilities, know who to contact and have practiced working together are better placed to coordinate a response.
This extends beyond organisational boundaries. As the discussions throughout the day highlighted, external networks and communities are valuable sources of shared experience. Learning from other organisations can help us challenge assumptions, discover different approaches and prepare for situations we may not have encountered ourselves.
There is no substitute for experience, but we can learn from the experience of others. The takeaway is clear, investing in resilience means investing in people as well as technology. We need to equip teams not only to build reliable systems, but also to respond confidently and collectively when those systems encounter disruption.
The day also highlighted the importance of making resilience part of everyday operations rather than treating it as a one-off project.
This means regularly reviewing recovery plans, validating assumptions, assessing dependencies and learning from incidents and exercises. It also means ensuring senior leaders understand the business implications and support the investment required to improve resilience.
Tools such as AWS Resilience Hub can support organisations in assessing applications against resilience policies, modelling potential weaknesses and reporting on areas that need attention. However, tools are most effective when they support a clear business strategy and are backed by the right people and processes.
Ultimately, resilience is not a fixed destination. Systems change, dependencies evolve and business priorities shift, our approach needs to evolve with them.
Perhaps the most useful challenge from the day was also one of the simplest, choose one workload and take a much deeper look at its resilience.
Start by understanding the business service it supports and the customer journeys that depend on it. Identify the stakeholders who need to be involved, explore how the service could fail and consider the consequences. Review the existing recovery arrangements, test the assumptions behind them and assess how well the team would respond to a realistic disruption.
This provides a manageable starting point for turning resilience from an abstract ambition into practical improvements.
It also creates an opportunity to look beyond the architecture. Does everyone understand their role? Can the team communicate effectively under pressure? Are decisions and escalation routes clear? Have people had the opportunity to practice? These questions are just as important as the technical ones.
The AWS Resilience Day, UK North reinforced that resilience is a shared responsibility, spanning technology, people, processes and leadership.
We cannot predict every disruption, and we should not assume that well-designed systems alone will protect us from every eventuality. What we can do is understand what matters most, prepare for a range of scenarios, practice responding under pressure and learn continuously.
When the unexpected happens, resilience is not just about whether our technology works, it is about whether our people are ready to respond and that readiness is something we need to build long before an incident begins.
If you'd like to explore how to build resilience into your projects from the outset, or how to prepare your teams for the failures they haven't yet imagined, we’d love to hear from you: https://www.leighton.com/lets-talk
Resilience is something every organisation knows it needs. But knowing it matters and being genuinely prepared when something goes wrong are two very different things.
That was one of the key messages from the inaugural AWS Resilience Day, which brought together members of the AWS UK tech community to explore how organisations can build, test and maintain resilience in an increasingly complex technology landscape.
Resilience is not just about building systems that never fail, it is about understanding what matters most to your business, preparing for disruption and giving your people the skills, processes and confidence to respond when the unexpected happens.
For organisations delivering critical services, the consequences of getting this wrong extend far beyond technology. They can affect customers, colleagues, business operations and, in some cases, essential services that people rely on every day.
Here are some of the key lessons we took away from the day.
One of the opening discussions, led by David Elliott, Director, Head of Technology at AWS, explored why resilience deserves greater attention at every level of an organisation.
He discussed a key challenge being that we tend to plan for the things we expect to happen. We consider familiar risks, assess the likelihood of disruption and put measures in place to manage them. But what about the events we haven't anticipated?
Rare, unexpected events – or ‘black swan’ events – can expose weaknesses that routine planning overlooks. These are precisely the moments when an organisation's resilience is put to the test.
Making the case for investment in resilience can be difficult, particularly when the benefits are not always immediately visible. However, the potential impact of a major disruption makes it a business-critical consideration, not simply a technical one.
This means starting with the business itself. What services are most important? Which customer journeys must be protected? How much disruption can the organisation tolerate, and what would the consequences be if a critical service became unavailable?
These questions need input from across the business, including technical teams, operational leaders and senior stakeholders. Without a shared understanding of priorities, it is difficult to make informed decisions about where to invest time and resources.
Chris Strug, a Senior Technical Account Manager from AWS reinforced an important principle – the goal isn't to create systems that never fail. It's to manage disruption effectively when failures inevitably occur.
Resilience responses involve three connected activities:
This is a useful way of thinking about resilience because it moves the conversation beyond prevention alone. Even with robust architecture and the appropriate safeguards, failures can still happen. What matters is how well an organisation anticipates, contains and recovers from them.
The same principle applies to disaster recovery and business continuity. Plans that were created several years ago may no longer reflect today's systems, dependencies, operating models or customer expectations. They need to evolve alongside the business.
Resilience should therefore be treated as a continuous lifecycle whereby objectives are established, appropriate measures are designed and implemented, they are then tested and evaluated, and organisations learn from the results before iterating. Crucially, testing and validation are essential. A plan that looks convincing on paper is not necessarily a plan that will work under pressure.
A further takeaway from the day was the importance of being realistic about resilience objectives.
Not every application, system or business process needs the same level of protection. Trying to make everything equally resilient can introduce unnecessary cost, complexity and operational overhead.
Instead, organisations should identify their critical business activities and the customer journeys that depend on them. Working with the right stakeholders, they can then assess what failure would mean, where the greatest risks sit and what level of disruption is acceptable.
This is where clear recovery objectives and service level objectives can help. They provide a practical way to define expectations, measure performance and understand whether resilience measures are delivering the intended outcomes.
The challenge is to balance business requirements with the realities of cost, complexity and operational effort. The right approach will look different for every organisation. The important thing is to make those decisions deliberately, based on business needs rather than technology alone.
One of the most valuable themes from the day was the importance of the human side of resilience. This was particularly relevant to the session delivered by our Leighton colleague, Mark Sailes.
Technology, architecture and documented processes all play an important role, but they are only part of the picture. When a serious incident occurs, people need to interpret incomplete information, make decisions quickly, communicate clearly and work together to resolve the problem.
These are not skills that can be developed solely by writing a disaster recovery plan or investing in more sophisticated technology, they need to be practiced.
You can read more about Mark’s talk here.
Under normal circumstances, teams have time to investigate, discuss options and make considered decisions. During a major incident, that luxury can disappear.
Pressure, uncertainty and competing priorities can make communication more difficult. People may focus on their own area of responsibility rather than the wider impact, or hesitate because they are unsure who should make a decision. Even experienced technical teams can find it challenging to maintain perspective when the situation is changing rapidly.
This is why the softer skills are so important. Clear communication helps teams share information and maintain a common understanding of what is happening. Collaboration helps people draw on different expertise rather than working in isolation. Leadership provides direction when there is uncertainty and good decision-making helps teams distinguish between what needs immediate action and what can wait.
Equally important is the ability to remain calm, adapt as new information emerges and recognise when an initial assumption may have been wrong.
These capabilities are not separate from technical resilience. They are what enable people to put technical capabilities and established processes into practice when they are needed most.
If we expect teams to perform well under pressure, we need to give them opportunities to practise before the pressure arrives. That means moving beyond checking whether a system can recover and testing how people respond when it does not behave as expected.
Scenario-based exercises and incident simulations can help teams practice handling uncertainty, communicating across different functions and making decisions with incomplete information. They can also reveal gaps in responsibilities, escalation routes and decision-making authority that might otherwise only become apparent during a real incident.
Exercises don't always need to be highly technical. A facilitated discussion around a realistic scenario can be just as valuable for exploring how teams share information, involve the right stakeholders and agree on priorities.
The aim is not to catch people out or expect every decision to be perfect. It is to create a safe environment in which teams can learn, build confidence and understand how they work together.
Afterwards, the debrief matters just as much as the exercise itself. What went well? Where did communication break down? Were decisions clear? Did people have the information and authority they needed? What would they do differently next time?
Those lessons should feed back into training, operational processes and future exercises.
Another important lesson is that resilience cannot sit with a single individual or technical team. A major incident may involve application teams, infrastructure specialists, operational colleagues, business leaders, suppliers and external partners. Each brings a different perspective, and effective recovery depends on those people working together.
Building relationships before an incident makes it easier to collaborate when one occurs. Teams that understand one another's responsibilities, know who to contact and have practiced working together are better placed to coordinate a response.
This extends beyond organisational boundaries. As the discussions throughout the day highlighted, external networks and communities are valuable sources of shared experience. Learning from other organisations can help us challenge assumptions, discover different approaches and prepare for situations we may not have encountered ourselves.
There is no substitute for experience, but we can learn from the experience of others. The takeaway is clear, investing in resilience means investing in people as well as technology. We need to equip teams not only to build reliable systems, but also to respond confidently and collectively when those systems encounter disruption.
The day also highlighted the importance of making resilience part of everyday operations rather than treating it as a one-off project.
This means regularly reviewing recovery plans, validating assumptions, assessing dependencies and learning from incidents and exercises. It also means ensuring senior leaders understand the business implications and support the investment required to improve resilience.
Tools such as AWS Resilience Hub can support organisations in assessing applications against resilience policies, modelling potential weaknesses and reporting on areas that need attention. However, tools are most effective when they support a clear business strategy and are backed by the right people and processes.
Ultimately, resilience is not a fixed destination. Systems change, dependencies evolve and business priorities shift, our approach needs to evolve with them.
Perhaps the most useful challenge from the day was also one of the simplest, choose one workload and take a much deeper look at its resilience.
Start by understanding the business service it supports and the customer journeys that depend on it. Identify the stakeholders who need to be involved, explore how the service could fail and consider the consequences. Review the existing recovery arrangements, test the assumptions behind them and assess how well the team would respond to a realistic disruption.
This provides a manageable starting point for turning resilience from an abstract ambition into practical improvements.
It also creates an opportunity to look beyond the architecture. Does everyone understand their role? Can the team communicate effectively under pressure? Are decisions and escalation routes clear? Have people had the opportunity to practice? These questions are just as important as the technical ones.
The AWS Resilience Day, UK North reinforced that resilience is a shared responsibility, spanning technology, people, processes and leadership.
We cannot predict every disruption, and we should not assume that well-designed systems alone will protect us from every eventuality. What we can do is understand what matters most, prepare for a range of scenarios, practice responding under pressure and learn continuously.
When the unexpected happens, resilience is not just about whether our technology works, it is about whether our people are ready to respond and that readiness is something we need to build long before an incident begins.
If you'd like to explore how to build resilience into your projects from the outset, or how to prepare your teams for the failures they haven't yet imagined, we’d love to hear from you: https://www.leighton.com/lets-talk
Resilience is something every organisation knows it needs. But knowing it matters and being genuinely prepared when something goes wrong are two very different things.
That was one of the key messages from the inaugural AWS Resilience Day, which brought together members of the AWS UK tech community to explore how organisations can build, test and maintain resilience in an increasingly complex technology landscape.
Resilience is not just about building systems that never fail, it is about understanding what matters most to your business, preparing for disruption and giving your people the skills, processes and confidence to respond when the unexpected happens.
For organisations delivering critical services, the consequences of getting this wrong extend far beyond technology. They can affect customers, colleagues, business operations and, in some cases, essential services that people rely on every day.
Here are some of the key lessons we took away from the day.
One of the opening discussions, led by David Elliott, Director, Head of Technology at AWS, explored why resilience deserves greater attention at every level of an organisation.
He discussed a key challenge being that we tend to plan for the things we expect to happen. We consider familiar risks, assess the likelihood of disruption and put measures in place to manage them. But what about the events we haven't anticipated?
Rare, unexpected events – or ‘black swan’ events – can expose weaknesses that routine planning overlooks. These are precisely the moments when an organisation's resilience is put to the test.
Making the case for investment in resilience can be difficult, particularly when the benefits are not always immediately visible. However, the potential impact of a major disruption makes it a business-critical consideration, not simply a technical one.
This means starting with the business itself. What services are most important? Which customer journeys must be protected? How much disruption can the organisation tolerate, and what would the consequences be if a critical service became unavailable?
These questions need input from across the business, including technical teams, operational leaders and senior stakeholders. Without a shared understanding of priorities, it is difficult to make informed decisions about where to invest time and resources.
Chris Strug, a Senior Technical Account Manager from AWS reinforced an important principle – the goal isn't to create systems that never fail. It's to manage disruption effectively when failures inevitably occur.
Resilience responses involve three connected activities:
This is a useful way of thinking about resilience because it moves the conversation beyond prevention alone. Even with robust architecture and the appropriate safeguards, failures can still happen. What matters is how well an organisation anticipates, contains and recovers from them.
The same principle applies to disaster recovery and business continuity. Plans that were created several years ago may no longer reflect today's systems, dependencies, operating models or customer expectations. They need to evolve alongside the business.
Resilience should therefore be treated as a continuous lifecycle whereby objectives are established, appropriate measures are designed and implemented, they are then tested and evaluated, and organisations learn from the results before iterating. Crucially, testing and validation are essential. A plan that looks convincing on paper is not necessarily a plan that will work under pressure.
A further takeaway from the day was the importance of being realistic about resilience objectives.
Not every application, system or business process needs the same level of protection. Trying to make everything equally resilient can introduce unnecessary cost, complexity and operational overhead.
Instead, organisations should identify their critical business activities and the customer journeys that depend on them. Working with the right stakeholders, they can then assess what failure would mean, where the greatest risks sit and what level of disruption is acceptable.
This is where clear recovery objectives and service level objectives can help. They provide a practical way to define expectations, measure performance and understand whether resilience measures are delivering the intended outcomes.
The challenge is to balance business requirements with the realities of cost, complexity and operational effort. The right approach will look different for every organisation. The important thing is to make those decisions deliberately, based on business needs rather than technology alone.
One of the most valuable themes from the day was the importance of the human side of resilience. This was particularly relevant to the session delivered by our Leighton colleague, Mark Sailes.
Technology, architecture and documented processes all play an important role, but they are only part of the picture. When a serious incident occurs, people need to interpret incomplete information, make decisions quickly, communicate clearly and work together to resolve the problem.
These are not skills that can be developed solely by writing a disaster recovery plan or investing in more sophisticated technology, they need to be practiced.
You can read more about Mark’s talk here.
Under normal circumstances, teams have time to investigate, discuss options and make considered decisions. During a major incident, that luxury can disappear.
Pressure, uncertainty and competing priorities can make communication more difficult. People may focus on their own area of responsibility rather than the wider impact, or hesitate because they are unsure who should make a decision. Even experienced technical teams can find it challenging to maintain perspective when the situation is changing rapidly.
This is why the softer skills are so important. Clear communication helps teams share information and maintain a common understanding of what is happening. Collaboration helps people draw on different expertise rather than working in isolation. Leadership provides direction when there is uncertainty and good decision-making helps teams distinguish between what needs immediate action and what can wait.
Equally important is the ability to remain calm, adapt as new information emerges and recognise when an initial assumption may have been wrong.
These capabilities are not separate from technical resilience. They are what enable people to put technical capabilities and established processes into practice when they are needed most.
If we expect teams to perform well under pressure, we need to give them opportunities to practise before the pressure arrives. That means moving beyond checking whether a system can recover and testing how people respond when it does not behave as expected.
Scenario-based exercises and incident simulations can help teams practice handling uncertainty, communicating across different functions and making decisions with incomplete information. They can also reveal gaps in responsibilities, escalation routes and decision-making authority that might otherwise only become apparent during a real incident.
Exercises don't always need to be highly technical. A facilitated discussion around a realistic scenario can be just as valuable for exploring how teams share information, involve the right stakeholders and agree on priorities.
The aim is not to catch people out or expect every decision to be perfect. It is to create a safe environment in which teams can learn, build confidence and understand how they work together.
Afterwards, the debrief matters just as much as the exercise itself. What went well? Where did communication break down? Were decisions clear? Did people have the information and authority they needed? What would they do differently next time?
Those lessons should feed back into training, operational processes and future exercises.
Another important lesson is that resilience cannot sit with a single individual or technical team. A major incident may involve application teams, infrastructure specialists, operational colleagues, business leaders, suppliers and external partners. Each brings a different perspective, and effective recovery depends on those people working together.
Building relationships before an incident makes it easier to collaborate when one occurs. Teams that understand one another's responsibilities, know who to contact and have practiced working together are better placed to coordinate a response.
This extends beyond organisational boundaries. As the discussions throughout the day highlighted, external networks and communities are valuable sources of shared experience. Learning from other organisations can help us challenge assumptions, discover different approaches and prepare for situations we may not have encountered ourselves.
There is no substitute for experience, but we can learn from the experience of others. The takeaway is clear, investing in resilience means investing in people as well as technology. We need to equip teams not only to build reliable systems, but also to respond confidently and collectively when those systems encounter disruption.
The day also highlighted the importance of making resilience part of everyday operations rather than treating it as a one-off project.
This means regularly reviewing recovery plans, validating assumptions, assessing dependencies and learning from incidents and exercises. It also means ensuring senior leaders understand the business implications and support the investment required to improve resilience.
Tools such as AWS Resilience Hub can support organisations in assessing applications against resilience policies, modelling potential weaknesses and reporting on areas that need attention. However, tools are most effective when they support a clear business strategy and are backed by the right people and processes.
Ultimately, resilience is not a fixed destination. Systems change, dependencies evolve and business priorities shift, our approach needs to evolve with them.
Perhaps the most useful challenge from the day was also one of the simplest, choose one workload and take a much deeper look at its resilience.
Start by understanding the business service it supports and the customer journeys that depend on it. Identify the stakeholders who need to be involved, explore how the service could fail and consider the consequences. Review the existing recovery arrangements, test the assumptions behind them and assess how well the team would respond to a realistic disruption.
This provides a manageable starting point for turning resilience from an abstract ambition into practical improvements.
It also creates an opportunity to look beyond the architecture. Does everyone understand their role? Can the team communicate effectively under pressure? Are decisions and escalation routes clear? Have people had the opportunity to practice? These questions are just as important as the technical ones.
The AWS Resilience Day, UK North reinforced that resilience is a shared responsibility, spanning technology, people, processes and leadership.
We cannot predict every disruption, and we should not assume that well-designed systems alone will protect us from every eventuality. What we can do is understand what matters most, prepare for a range of scenarios, practice responding under pressure and learn continuously.
When the unexpected happens, resilience is not just about whether our technology works, it is about whether our people are ready to respond and that readiness is something we need to build long before an incident begins.
If you'd like to explore how to build resilience into your projects from the outset, or how to prepare your teams for the failures they haven't yet imagined, we’d love to hear from you: https://www.leighton.com/lets-talk
Resilience is something every organisation knows it needs. But knowing it matters and being genuinely prepared when something goes wrong are two very different things.
That was one of the key messages from the inaugural AWS Resilience Day, which brought together members of the AWS UK tech community to explore how organisations can build, test and maintain resilience in an increasingly complex technology landscape.
Resilience is not just about building systems that never fail, it is about understanding what matters most to your business, preparing for disruption and giving your people the skills, processes and confidence to respond when the unexpected happens.
For organisations delivering critical services, the consequences of getting this wrong extend far beyond technology. They can affect customers, colleagues, business operations and, in some cases, essential services that people rely on every day.
Here are some of the key lessons we took away from the day.
One of the opening discussions, led by David Elliott, Director, Head of Technology at AWS, explored why resilience deserves greater attention at every level of an organisation.
He discussed a key challenge being that we tend to plan for the things we expect to happen. We consider familiar risks, assess the likelihood of disruption and put measures in place to manage them. But what about the events we haven't anticipated?
Rare, unexpected events – or ‘black swan’ events – can expose weaknesses that routine planning overlooks. These are precisely the moments when an organisation's resilience is put to the test.
Making the case for investment in resilience can be difficult, particularly when the benefits are not always immediately visible. However, the potential impact of a major disruption makes it a business-critical consideration, not simply a technical one.
This means starting with the business itself. What services are most important? Which customer journeys must be protected? How much disruption can the organisation tolerate, and what would the consequences be if a critical service became unavailable?
These questions need input from across the business, including technical teams, operational leaders and senior stakeholders. Without a shared understanding of priorities, it is difficult to make informed decisions about where to invest time and resources.
Chris Strug, a Senior Technical Account Manager from AWS reinforced an important principle – the goal isn't to create systems that never fail. It's to manage disruption effectively when failures inevitably occur.
Resilience responses involve three connected activities:
This is a useful way of thinking about resilience because it moves the conversation beyond prevention alone. Even with robust architecture and the appropriate safeguards, failures can still happen. What matters is how well an organisation anticipates, contains and recovers from them.
The same principle applies to disaster recovery and business continuity. Plans that were created several years ago may no longer reflect today's systems, dependencies, operating models or customer expectations. They need to evolve alongside the business.
Resilience should therefore be treated as a continuous lifecycle whereby objectives are established, appropriate measures are designed and implemented, they are then tested and evaluated, and organisations learn from the results before iterating. Crucially, testing and validation are essential. A plan that looks convincing on paper is not necessarily a plan that will work under pressure.
A further takeaway from the day was the importance of being realistic about resilience objectives.
Not every application, system or business process needs the same level of protection. Trying to make everything equally resilient can introduce unnecessary cost, complexity and operational overhead.
Instead, organisations should identify their critical business activities and the customer journeys that depend on them. Working with the right stakeholders, they can then assess what failure would mean, where the greatest risks sit and what level of disruption is acceptable.
This is where clear recovery objectives and service level objectives can help. They provide a practical way to define expectations, measure performance and understand whether resilience measures are delivering the intended outcomes.
The challenge is to balance business requirements with the realities of cost, complexity and operational effort. The right approach will look different for every organisation. The important thing is to make those decisions deliberately, based on business needs rather than technology alone.
One of the most valuable themes from the day was the importance of the human side of resilience. This was particularly relevant to the session delivered by our Leighton colleague, Mark Sailes.
Technology, architecture and documented processes all play an important role, but they are only part of the picture. When a serious incident occurs, people need to interpret incomplete information, make decisions quickly, communicate clearly and work together to resolve the problem.
These are not skills that can be developed solely by writing a disaster recovery plan or investing in more sophisticated technology, they need to be practiced.
You can read more about Mark’s talk here.
Under normal circumstances, teams have time to investigate, discuss options and make considered decisions. During a major incident, that luxury can disappear.
Pressure, uncertainty and competing priorities can make communication more difficult. People may focus on their own area of responsibility rather than the wider impact, or hesitate because they are unsure who should make a decision. Even experienced technical teams can find it challenging to maintain perspective when the situation is changing rapidly.
This is why the softer skills are so important. Clear communication helps teams share information and maintain a common understanding of what is happening. Collaboration helps people draw on different expertise rather than working in isolation. Leadership provides direction when there is uncertainty and good decision-making helps teams distinguish between what needs immediate action and what can wait.
Equally important is the ability to remain calm, adapt as new information emerges and recognise when an initial assumption may have been wrong.
These capabilities are not separate from technical resilience. They are what enable people to put technical capabilities and established processes into practice when they are needed most.
If we expect teams to perform well under pressure, we need to give them opportunities to practise before the pressure arrives. That means moving beyond checking whether a system can recover and testing how people respond when it does not behave as expected.
Scenario-based exercises and incident simulations can help teams practice handling uncertainty, communicating across different functions and making decisions with incomplete information. They can also reveal gaps in responsibilities, escalation routes and decision-making authority that might otherwise only become apparent during a real incident.
Exercises don't always need to be highly technical. A facilitated discussion around a realistic scenario can be just as valuable for exploring how teams share information, involve the right stakeholders and agree on priorities.
The aim is not to catch people out or expect every decision to be perfect. It is to create a safe environment in which teams can learn, build confidence and understand how they work together.
Afterwards, the debrief matters just as much as the exercise itself. What went well? Where did communication break down? Were decisions clear? Did people have the information and authority they needed? What would they do differently next time?
Those lessons should feed back into training, operational processes and future exercises.
Another important lesson is that resilience cannot sit with a single individual or technical team. A major incident may involve application teams, infrastructure specialists, operational colleagues, business leaders, suppliers and external partners. Each brings a different perspective, and effective recovery depends on those people working together.
Building relationships before an incident makes it easier to collaborate when one occurs. Teams that understand one another's responsibilities, know who to contact and have practiced working together are better placed to coordinate a response.
This extends beyond organisational boundaries. As the discussions throughout the day highlighted, external networks and communities are valuable sources of shared experience. Learning from other organisations can help us challenge assumptions, discover different approaches and prepare for situations we may not have encountered ourselves.
There is no substitute for experience, but we can learn from the experience of others. The takeaway is clear, investing in resilience means investing in people as well as technology. We need to equip teams not only to build reliable systems, but also to respond confidently and collectively when those systems encounter disruption.
The day also highlighted the importance of making resilience part of everyday operations rather than treating it as a one-off project.
This means regularly reviewing recovery plans, validating assumptions, assessing dependencies and learning from incidents and exercises. It also means ensuring senior leaders understand the business implications and support the investment required to improve resilience.
Tools such as AWS Resilience Hub can support organisations in assessing applications against resilience policies, modelling potential weaknesses and reporting on areas that need attention. However, tools are most effective when they support a clear business strategy and are backed by the right people and processes.
Ultimately, resilience is not a fixed destination. Systems change, dependencies evolve and business priorities shift, our approach needs to evolve with them.
Perhaps the most useful challenge from the day was also one of the simplest, choose one workload and take a much deeper look at its resilience.
Start by understanding the business service it supports and the customer journeys that depend on it. Identify the stakeholders who need to be involved, explore how the service could fail and consider the consequences. Review the existing recovery arrangements, test the assumptions behind them and assess how well the team would respond to a realistic disruption.
This provides a manageable starting point for turning resilience from an abstract ambition into practical improvements.
It also creates an opportunity to look beyond the architecture. Does everyone understand their role? Can the team communicate effectively under pressure? Are decisions and escalation routes clear? Have people had the opportunity to practice? These questions are just as important as the technical ones.
The AWS Resilience Day, UK North reinforced that resilience is a shared responsibility, spanning technology, people, processes and leadership.
We cannot predict every disruption, and we should not assume that well-designed systems alone will protect us from every eventuality. What we can do is understand what matters most, prepare for a range of scenarios, practice responding under pressure and learn continuously.
When the unexpected happens, resilience is not just about whether our technology works, it is about whether our people are ready to respond and that readiness is something we need to build long before an incident begins.
If you'd like to explore how to build resilience into your projects from the outset, or how to prepare your teams for the failures they haven't yet imagined, we’d love to hear from you: https://www.leighton.com/lets-talk
