Author: Vinay Nagpal, Associate at Portman Partners
The Internet is ubiquitous and an increasingly integral part of our lives. Before we get into the hidden attributes of the Internet and the risks they pose, let’s start with the basics. The strength of TCP/IP and hence the Internet is that by design it relies on a distributed architecture for resiliency and security. When Vint Cerf co-invented the Internet over 50 years ago, the basis of the TCP/IP protocol was packet-switching, as opposed to circuit-switching.
When you are speaking on a telephone (“land” line), you are using circuit-switching, as the entire circuit is “engaged” for the duration of the call. Comparing that, for example, to when you are sending an email, Instagram message, Twitter, Facebook post, or for that matter reading this Blog, you are using packet-switching wherein the data is broken down into small chunks of packets sent to its destination via different routes. Upon arrival, the packets are re-assembled in sequential order and we see the message in its entirety. However, as stated, the various packets in their journey from the source to the destination take different paths over the Internet to reach their end destination. Prior to the explosion of data that has taken place over the last few years, packet-switching was accomplished with relatively few network access points as the levels of data traffic were well served by a small set of global hubs due to the nature of the packet and message size – i.e. these were text-based messages prior to the app-based revolution.
Vint Cerf, Chief Internet Evangelist of Google and Co-Inventor of the Internet had this to say about the importance of diversity of Internet Infrastructure and economic opportunity that it brings:
“Economic opportunity doesn’t just happen, it happens because people think about what is needed, what ingredients are required in order to create fertile territory in order for the growth of new businesses, and I can tell you that among the ingredients Infrastructure is an important part, especially if you are going to build a business that has anything to do with the Internet.”To expand on this, every business is now an Internet reliant business whether it is local or global in nature.
These different “paths” taken by packets are enabled by routing that allows one network to talk to another network. Hence the Internet is truly a “network of networks”. The distributed nature of the Internet is what makes it scalable, resilient, and in its lifecycle over the last five decades has transformed into the “fourth utility” – after electricity, water and natural gas. Of the 7.3 billion people on our planet, last year the world’s Internet users passed the 4 billion mark. When users are accessing the Internet online, they are accessing data, data which is residing in a secure, reliable, highly-connected building called a data centre. These data centres are sophisticated buildings and complex infrastructure filled with computers, servers, power plants, chillers and other machinery that consumes tremendous amounts of power and water. This mass consumption of resources in a concentrated set of buildings has a dramatic impact on the region’s infrastructure including the ability to service critical resources such as hospitals, emergency services, housing, office buildings and other important assets.
The data center industry, which is still a relatively unknown industry to many, even in the technology sector, has been growing at an exponential pace and therefore consuming more than its fair share of the region’s utilities to the detriment of the region. According to various industry reports, over the next four years from 2019 to 2023, the data centre industry is expected to grow between 15-17% CAGR globally. This growth needs to be paired in parallel with the growth in underlying infrastructure and utilities required to produce and transport data from one part of the world to the other and to get the data ultimately from its “home” – which most likely is a data centre somewhere in the world to the end-user. During the data journey, there are critical parts of underlying infrastructure involved including subsea cables, terrestrial fibre cables, Internet Exchanges (IX’s), cell towers, utility substations and more. In addition to this, for a variety of reasons, data increasingly needs to be transferred in the most efficient, economical, secure and fastest means. As the Internet grew, these four core critical components took a back seat to an industry that continued to use access points already established as an easy path for growth. In this document, we will examine why this has a severe negative effect on both data and regional economies and their supporting utility infrastructure. Finally, we will draw a conclusion if the path we are on is the right one for the future of the Digital Economy. Did Amsterdam get it right or wrong?
Not surprisingly, the concentration of data centres has naturally gravitated towards the high populous centres of the world and those that grew near the early technology adopters/companies that needed the Internet most. Typically, populous centres have access to highly skilled labour, abundant land, access to utilities – electricity, water and fibre, proximity to airports and of course a myriad of other factors including tax incentives put together by state, county, municipalities and/or other jurisdictions. This concentration of data centres in a particular geography certainly leads to socio-economic benefits for that region, but it also taxes the non-infinite utilities – be it land, electricity, talent, water or right of ways on public roads required to dig up the streets and install terrestrial fibre cables. As mentioned, it also further exasperates the problem of foregoing the 50+ year old philosophy and protocol requirement of diversity and resiliency. In the second example, early Internet company adopters used none of this thinking to select their locations leading to current Internet hubs that have outgrown their purpose and inhibit the organic growth of the Internet where it is needed most.
Hindsight is 20/20 and we certainly didn’t fully understand the future growth of the Digital Economy when we came up with the Internet’s design while we connected with 9.6kbps modems – once thought to be the most bandwidth we would ever need to communicate in that Era of the digital boom. Remember, 30 years ago our messages were text-based and there was no World Wide Web or myriad of Social Media sites driving not only text but pictures, movies and a wide range of other related bandwidth-consuming content. This flaw in thinking, unknown at the time, did not predict the problems of so few Internet hubs.
The realization of a lack of diversity with infrastructure started with Hurricane Sandy hitting the NE USA which crippled the Internet.
Recent blackouts in NYC just weeks ago proved that we can’t always learn from the past to protect the future. In addition, the reliance on a few subsea landing cable locations exposes the world’s digital economy and is viewed to be a vulnerability to malicious intent. Throw in limited supply of resources for regional utility infrastructure and the impact of locations susceptible to weather conditions and the stress placed on any region leads to instability. So we ponder how to alleviate issues by expanding diversity globally to offset the obvious pending issues at hand. This endeavour will take years not weeks or months, but the energy devoted to this effort must begin now or we will face catastrophic events in the future. Events that will impact every person and business on the planet as their lives and business models more and more revolve around a stable and secure Internet. We can choose to stay the course and adjust when catastrophe hits or get ahead of the problem and build a resilient network that predicts and avoids catastrophic events. The choice is obvious, but it takes awareness of the issue from the masses to create and maintain change.
Amsterdam, which is one of the most “concentrated data centre markets”, is part of what is referred to as the FLAP region in Europe (Frankfurt, London, Amsterdam and Paris). These have been the four key hubs in Europe as it pertains to data centre and underlying Internet Infrastructure. Amsterdam is home to over 30% of the Data Centers in Europe. The subsea industry has been diligently and pro-actively working towards diversification of the landing of subsea cables as noted recently with new subsea cables landings in Europe in Sopelano, Spain (MAREA cable), Saint-Hilaire-de-Reiz, France (DUNANT cable), Lisbon, Portugal (Equiano cable) and Eemshaven Netherlands (COBRA cable). In addition, the United States has a concentrated group of ports where these cables land.
They are NY/NJ, Miami, Los Angeles, and the Pacific Northwest. What was not foreseen or even thought to ever be likely, was a catastrophic or targeted malicious event that could jeopardize the flow of Internet traffic on these routes and thus take the Internet down or cause significant damage to those businesses and consumers relying on them. The standard operating procedure has been to react to one of these events rather than predict it. This changed with Hurricane Sandy in 2012 impacting the NE United States region and impacting the global financial industry. The NYSE shut down for two days, emergency response communication was down for several days and power was affected for a prolonged period. It set off a long-term evaluation of how to diversify the landing sites of subsea cables and search out new locations immune, as much as possible, to Mother Nature and malicious intent. Related very closely to this was the idea that Data Center growth in concentrated areas faced similar issues. The concept of where to land the subsea cables MAREA and BRUSA were rethought as a result of Hurricane Sandy and in turn, led to the selection of VA Beach as the chosen landing site. This was the first of many decisions that would be based on areas previously overlooked.
On Monday, July 15th, 2019, the Amsterdam Metropolitan Government placed a stoppage on new data centre builds. To many, this was an “unexpected” data centre stoppage. However, it represented the first action to be based on a future catastrophe vs a current catastrophic event. It’s very understandable given the years of study behind it and the evaluation of past events. The Amsterdam Metropolitan Area (AMA) announced a stoppage to the establishment of new data centres in the region including Amsterdam and Haarlemmermeer. As quoted by an Amsterdam government official: “Data centres in the Amsterdam area take up a lot of territory space while they place a high burden on the electricity grid. It would be a temporary halt, but the government already dropped some hard-line demands about gaining government control over data centre deployments and demanding free supply of residual heat for the heating of houses.”
Recently, Ashburn suffered a 6-hour outage when two peering nodes failed causing a widespread outage of popular websites impacting businesses as far away as Germany. Meanwhile, the biggest global data center operators (in Ashburn) have managed to pull most of the needed ecosystem into their sites and are capable of shutting out the rest of the industry unless they pay exuberant fees to access these services and keep their customers online transacting business as normal. It doesn’t matter that these data centres don’t have the space to accommodate all companies wanting to colocate with them. The goal is to put a wall around the ecosystem and show Wall Street increased margin in services while they tax all those trying to reach their environment. This runs contrary to the idea that the Internet is an open environment and not to be controlled by a select few companies and that cross-connect pricing should be consistent and transparent to companies of all sizes. Should this manifest, it will next lead to higher colocation prices at a time when enterprises are trying to divest of company-owned data centers and move into more efficient infrastructure run by experts while the enterprises turn their attention to core business operations. However, enterprises are blind to infrastructure and its geographic necessity.
Instead, the current enterprise focus is on how to create a killer app for commerce and preserve their ranking in their respective vertical. Blindness to infrastructure creates a huge problem. Imagine a Bank having a stellar reputation and being recognized as having a leading-edge app for its consumers. Without the right geographic diversity and underlying expansion of infrastructure in new regions – both of their focuses will fail and hence the bank will fail. We can’t bank online if apps don’t work or are slow or only reach those customers in a select few data centres. In addition, inefficient and costly Internet paths, by way of a very few network access points, will further jeopardize any businesses’ ability to grow and retain customers.
Instead, they will lose customers and be forced to do business as they did 20 years ago – which means they will go out of business. This forced strategy by a handful of data centre operators cost more and halts innovation. Under these circumstances, the Internet will cease to be what early pioneers such as Vint Cerf, Bob Khan and Heidi Heiden imagined for it. Enterprises working with the monopolies which control the Internet will ultimately slow global business growth, increase cost and paralyze innovation. Customers need champions, such as the IEIC, to represent their needs, protect their rights and create change.
When thinking about the future of the Internet we need to focus on dense and diverse fibre routes living in harmony with large underutilized utilities. Through this lens you can imagine new regions becoming major players in the global digital economy such as Henrico County has done with the new Network Access Point (Richmond NAP home to the MAREA and BRUSA subsea cables). This philosophy also leads to more efficient, faster, resilient and affordable data distribution. It also paves the way to develop new regions prospering from the Internet boom related to the Data Economy – in a sense helping to reinvent their regions by attracting workers, businesses and creating a robust set of offshoot economic development areas such as construction, housing, retail, schools, hospitals and more.
In conclusion, if we don’t begin to think about and invest in the Internet ‘highway’ and associated communities and workforce we will be setting ourselves up for a future that ultimately will lead to disastrous outcomes.
A recent example took place in Dublin in 2016. A suburb of Dublin, Grange Castle, which enticed data center companies to come to the region via various incentives actually ran out of utility grid power four years earlier than projected. This led to piping in natural gas from neighbouring regions to produce the needed MWs to power data centres. Sounds ok at first, but when transporting the natural gas, the transmission caused a 60% degradation in delivered power. This meant if you needed to produce 10MWs you had to pay and transport 40MWs in addition to having to build the gas pipeline infrastructure and build a natural gas substation far enough away in accordance with the county’s noise ordinance.
All in all, it increased cost and complexity and created a four-year timeline for Grange Castle to get their new power plant online, which is ongoing as we write this. What else happened to the region when utility power ran out? The cost to provide energy to homes went up, Hospital construction was put on hold, schools could not be opened, construction declined. Other devasting impacts emerged to hit the residents of Grange Castle which enjoyed four years of growth and associated revenue from Data Centers. Once affordable and reliable power was gone it created a decline in property value, greater expense to run households and more. All because of a short-lived euphoria that ultimately only served to throw the county into a four-year depression. The right answer? When Grange Castle reached a point at which they knew they were in trouble with regard to limited resources such as power, water and available land, they should have instituted a policy to limit the industry from causing such long-term damage to their community. And perhaps this use-case was evaluated by Amsterdam to understand the downside of ‘excess and greed’ and how the pursuit to join the Digital Economy without a plan will, not might, ultimately end poorly.
Today there are approximately 15 global Internet hubs and we argue that there should be many more. The development of additional regional Internet hubs will change economies that have suffered much like Pittsburgh which lost their steel business and St. Louis which is trying to re-invent itself as a digital economy attracting people, companies, innovation and most of all create a healthy economic region. The diversification of the Internet requires prominent loud voices, time, awareness, industry agreement and strong educated policy makers in cities around the globe. Finally, you cannot overlook the economic benefit to the world of having more communities participating in what is sure to be a growth engine of economies for the next 50 years or more.
The Amsterdam Effect, what does it mean? Perhaps it will go ignored. More likely, it will force thoughtful conversation around what is good for dense technology regions and what needs to be done to avoid catastrophic events in the future where the Internet relies on such few interconnection points. Will other major regions follow suit? As mentioned, Frankfurt and Ashburn face these concerns and will have to make a decision. Follow Amsterdam’s example or continue to build out until power, talent, water and land become extinct – and with that extinction, what will be the impact on the Internet and those of us that rely on it. The IEIC will work tirelessly to ensure that these and other regions follow the lead of Amsterdam to protect the Internet, grow new regions and ensure the global platform is one that the businesses can depend on.
This blog was originally co-written by Vinay Nagpal for Inter Globix. You can find the original blog here.