Home Office Immigration Technology

How the Home Office’s Immigration Technology department reduced its cloud costs by 40%

Identity E2E have been leading the Home Office EBSA platform team for the past 4 years, working with other suppliers to deliver a large scale enterprise capability for hosting its applications on the AWS public cloud.

The Home Office team use the cloud as their primary hosting platform and the department is one of the biggest cloud users in the UK government. The Immigration Technology department has over 500 developers working on immigration projects.

Home Office senior leaders wanted Immigration Technology to reduce spending on the cloud, so its platform team investigated ways to reduce costs over the last year.

Benefits of using the cloud more efficiently

Immigration Technology reduced its overall cloud costs by 40%, by using a variety of optimisation techniques across storage, use and resources. By continuing these techniques, the team is confident it can increase cloud cost savings by at least another 20% as they continue to experiment.

How the Home Office cut cloud costs

The team looked at how developers were working and found they were not using cloud resources as efficiently as they could because of their focus on delivery. By focusing on mission-critical deadlines, developers were using more expensive on-demand services and not planning resource usage. This led to expensive cloud bills, sometimes called “bill shock”, because most cloud providers charge for cloud resources by the second.

Immigration Technology researched industry practices to help reduce cloud costs and discovered 7 strategies aimed at increasing efficiency.

1. Using excess capacity in the cloud

Potential cost saving: the department paid approximately 80% less for cloud resources by using excess capacity services.

2. Scheduling services

Potential cost saving: Immigration Technology reduced the costs by over 60% by turning off services overnight and over the weekend without affecting user functionality.

3. Using autoscaling

Immigration Technology encouraged teams to use autoscaling, but not all developers were using this method. Most cloud providers offer an autoscaling product. Immigration Technology decided to incorporate one as a standard component into their build templates, which means the department automatically has autoscaling as part of every standard build.

4. ‘Rightsizing’ components

After analysing its cloud estate, Immigration Technology discovered that environments did not always have the appropriate compute and storage resources.

For example, the team monitored some of their compute instances over a 2-week period and found they were using only 10% to 20% of the resources they were paying for on average. Immigration Technology wanted to get its compute usage between 60% to 80% to get better value for money.

To maximise usage, Immigration Technology set standard build templates to scale services appropriately for their environments, a strategy called ‘rightsizing’.

Potential cost savings: By ‘rightsizing’ services, Immigration Technology kept costs low in non-production and other low usage environments.

5. Re-architecting the service

In some cases, the platforms team found the best way to reduce costs was to re-architect the service to make it cloud native. For example, Immigration Technology found that folding multiple databases into one managed database instance helped to reduce costs considerably. They also moved applications into containers to reduce the resource footprint of the services

Immigration Technology also made another big change to reduce its compute footprint. The team switched some services that did relatively simple tasks to use serverless technology instead. They also replaced commercial middleware tools with simpler functions, such as using storage lifecycle policies instead of a more expensive document management tool.

Potential cost savings: Immigration Technology used cheaper cloud native options by re-architecting services that could not be made more efficient.

6. Performing housekeeping tasks

Development teams were using a large number of cloud resources during testing or building, but did not always stop or delete them once testing was complete. Immigration Technology saved 1% in costs by cutting unused resources and encouraging development teams to keep their environments tidy.

Immigration Technology development teams regularly perform housekeeping tasks like:

• deleting old snapshots and backups
• deleting environments or hosts that are no longer required
• creating lifecycle policies to manage data
• deleting old database instances that no longer have any connections
• removing old data storage volumes that are no longer attached to servers

Potential cost savings: Doing regular housekeeping tasks and creating policies to archive older assets saved around 1% on cloud costs.

7. Up-front usage commitments

Most cloud providers will offer a discount in return for committing to specified level of usage. Sometimes these discounts are referred to as ‘reserved’ or ‘committed use’ instances. Immigration Technology found they could get an upfront discount of up to 40% on compute resources, if they bought in bulk upfront.

However, up-front usage commitments are often restricted to a specific server type and the server must be switched on 24 hours a day, 7 days a week. This strategy cannot be used alongside other cloud cost saving techniques like scheduling where servers are switched off for a period of time (Scheduling).

Immigration Technology also found the savings from the up-front usage commitments were often lower than from many of the other techniques. For example, turning off (Scheduling) servers overnight and at weekends can save over 60% on cloud costs. Immigration Technology realised it could save more money by only using this method for services where no other techniques would work.

Potential cost savings: Immigration Technology only considered up-front usage commitments if no other technique would have worked.

Related Link

How the Home Office’s Immigration Technology department reduced its cloud costs by 40

About IdentityE2E

IdentityE2E is a specialist SME, recognised experts in the field of identity management, biometrics and the integration of enterprise scale cloud solutions. IdentityE2E are an AWS Partner who specialise in providing cloud engineering solutions to private and public sector customers.

Contact

Mat Costick, Email: info@identityE2E.com