The Amazon cloud will Auckland Council slash the processing time required to process hundreds of terabytes of data from months to just two days
The Smart Growth Portal real time data tool powered by The Instillery and the Amazon Web Services (AWS) Cloud will enable the council to make better, and more timely planning decisions.
Due to be launched later this month, the AWS Cloud will enable the council to analyse massive amounts of data using the most secure platform and advanced technology in a 24/7 environment, thereby enabling the council to provide more meaningful data for planning to stakeholders in much less time than was previously the case.
Acting as a large analytical hub, the tool connects all of Auckland Council’s existing data sets, including those of its infrastructure providers; Auckland Transport and Watercare Limited, to create a world-first unique 2D and 3D connected, interactive view of Auckland.
Dr Haydn Read, Head of Infrastructure Programmes for the Development Programme Office (DPO) at Auckland Council, says consolidating the city’s infrastructure assets in one place provides an intuitive storytelling platform for city stakeholders and citizens to engage on various topics in order to debate and make more informed decisions.
To achieve this, the Instillery solution for The Auckland City Council included a range of AWS Cloud services, such as Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3), and Amazon Sagemaker to enable the building of machine learning models, and Amazon Elastic MapReduce (Amazon EMP).
“By connecting the natural, built and social environments into a realistic digital version of Auckland, overlaying resilience data sets and providing the ability to develop scenarios we can provide deeper analysis to help inform and solve the investment discussions and decisions in areas such as housing and roads,” Read says.
The Smart Growth Portal was designed by Auckland Council, which worked exclusively with The Instillery to deliver a secure, highly scalable, automated and economic cloud platform built on the AWS Cloud.
The Instillery founder & CEO Mike Jenkins says a major challenge the project faced was the time it took the tool to process the large amounts of data required.
“Previously it was taking weeks, sometimes months, to crunch some of the larger data sets – literally grinding their traditional servers to a halt,” he explains.
“Given the time required to perform the analytics and a lack of real data, decisions would need to be delayed. It’s great to see Auckland Council investing in cloud consumption models from AWS where it’s only paying for IT resources used when processing analytics.”
Read asserts that Smart Growth Portal is not just about offering a digital picture of what’s happening in Auckland and its’ communities now – it’s about looking well into the future and analysing the data in a dynamic sense.
“By mapping the roads, for instance, with public transport routes and overlaying that data with social indicators such as population, health, unemployment and income, we can start to transform the data into highly valuable information to see what those communities might look like in the future and where we should prioritise investment.
“The Instillery and the AWS Cloud play a key role in enabling this to happen.”
Read further notes” “We opted to build the Smart Growth Portal platform on the AWS Cloud because it enabled pace and scale.
“By taking advantage of the innovative architecture design by The Instillery powered by the AWS Cloud, we’ve been able to reduce the time it takes to process hundreds of terabytes of data from months to just two days.
“We can now analyse huge amounts of data using the most advanced technology in a 24/7 environment, enabling us to run an analytic at the end of the business day and have the answers with data points for stakeholders the following day.”
Read says there is a “very real” opportunity for the Smart Growth Portal to expand and accommodate other data systems – ensuring it can be scaled to adopt central and local government agencies data to glean better insights.
MBIE, Housing New Zealand and a number of other agencies have already indicated their interest in leveraging the portal.
“We’re excited about what the Smart Growth Portal can offer New Zealand in terms of using big data analytics to guide decision making, inform investment and offer insight to the social outcomes those decisions need to deliver,” Read says.
Tim Dacombe-Bird, AWS Country Manager for New Zealand, says; “New Zealanders have always had an innate ability to solve problems in new ways.
“This groundbreaking work by the Auckland City Council and AWS Advanced Consulting Partner and Authorised Public Sector Partner, The Instillery, is yet another example of that innovative approach.
“Leveraging the power of the AWS Cloud to help improve services to citizens in New Zealand’s largest city, is an outstanding outcome.”
