Data storage: the third big challenge of IT

The development of information technology over the past decades saw several phases, each with its own key challenge. Twenty years ago or so, processing power was the dominant topic, with Moore’s Law at the heart of technology debates. With the rise of networked computing and the internet, available bandwidth took centre stage. Both challenges - processing power and network bandwidth - still get their fair share of technical progress, but in business, their improvement isn’t as pressing as it used to be a few years ago.

Today, organisations face a third big challenge that awaits tackling in the coming years; in the face of exponentially increasing amounts of data, now storage space becomes a critical factor.

Companies not only have to cope with more data from more sources. Another problem arises at the far end of the business processes, where a second mountain of data is growing at least as rapidly: each byte of production data needs at least another byte of corresponding storage. Often, it’s a multiple of that, resulting firstly from the requirement of keeping data available anywhere all the time, secondly from the need for backup copies.

If an organisation has a data recovery strategy - which is highly recommended - a single copy of the data is not sufficient. As a consequence, 10 terabytes of production data adds up to at least another 10 plus X terabytes in backup data or more, depending on the recovery point objectives of the organisation. This gives the IT infrastructure and its architects three challenges to tackle which again sound quite familiar: reduce data volume, save time and provide enough network bandwidth.

Here are three tips on how to tackle these challenges:

1. Deduplication and compression

Keep your copies and storage volume as small as possible. When talking about copies of production data, deduplication and compression are the key technologies to save space on your data stores and avoid unnecessary investment in expensive hardware. What is more, this is also true for the networks that carry the backup data. Beefing them up to 10GbE standard is very expensive, especially if that happens purely to accelerate backups. So it makes more sense to reduce the volume of data to be transferred before it goes over the network. The advantages of deduplication and data compression multiply with the use of backup in the cloud, where the bandwidth of the WAN link is the limiting factor. Finally, state-of-the-art backup solutions offer changed block tracking to further reduce the volume of data to be stored and transferred.

2. Cut the backup process short

With increasing volumes of data, time becomes a scarce resource. The more data to backup, the more time you will need. An architecture that leverages several backup proxies can accelerate the process by enabling a backup server to manage multiple backup streams simultaneously.

When considering data restores, recovery time can be reduced by technologies that allow for individual files to be recovered back to production without touching the entire backup image.

3. Don’t forget ease of use and simplicity

A final tip to close this article: solutions that are simple to set up and easy to use will make your life as an administrator easier in the long run. Modern data centres tend to become more and more complex over time, and making basic processes like data protection fast and as automated as possible will give you more time and budget to take care of what really matters: innovative projects.

As you will realise, the simple equation ‘more data = more storage’ doesn’t do the problem justice as, for most organisations, the continuous adding of more storage space would be too expensive and complex an effort to take on. The solution, however, lies in the smart use of virtualised infrastructures and corresponding technologies. They can help achieve more with less.

*Charles Clarke is Technical Director, APAC at Veeam Software.