Revitalising IT harmony

We are all familiar with the ‘tragedy of the commons’, where multiple individuals, acting independently and rationally, consume a shared resource (like clean drinking water and fisheries), leading to its degradation or depletion. This concept highlights the challenge of managing and conserving common resources in the face of individual self-interest. It suggests that collective action through establishing rules, regulations or institutions is often necessary to prevent the tragedy of the commons and ensure the sustainable use of shared resources.

The modern IT environment has given us a new case to consider related to the tragedy of the commons: data. Data, like water, is a critical resource. Developers want to use data extensively to build innovative applications and products. However, excessive data in a distributed enterprise environment can pose risks to any large business.

The data developers use often includes customers’ personal data, such as names and birth dates. The risks of damage or the information being stolen can be catastrophic. This is where database administrators (DBAs) are tasked with managing and securing this data. There is an inherent tension between developers who want as much access as possible and the DBAs that want to limit that access. How data is used within an enterprise is critical because it must be used to create new solutions and be kept secure. An overly open data policy could lead to breaches and leaks, while an overly restrictive policy limits innovation.

Thankfully, there is a solution to help ease this tension. Observability can automatically analyse massive amounts of information across a business’s entire IT environment. Users can use it to pinpoint the causes of outages or performance issues and receive actionable insights to resolve these problems quickly. Modern observability solutions even use AI to predict and proactively prevent problems before they occur. DBAs and developers can utilise observability to strike a balance while boosting innovation and improving productivity. Here are the three most significant ways that observability can solve the tragedy of the data commons and ease the feud between DBAs and developers.

Safeguarding information

One of the biggest concerns DBAs have regarding data access is security. Sensitive production data is no longer stored exclusively on a company’s on-premises servers, where DBAs have complete visibility into how it is being used and who has access. Instead, it is now stored across various private and public clouds and on-premises, reducing visibility for DBAs.

This sensitive data is the same information developers often need to build and launch applications in a continuous delivery model, where developers build software in short cycles to release at any time. This involves working across different clouds, including AWS and Azure while leveraging complicated cloud-enabled technologies like microservices and Docker containers. When data is deployed across multiple clouds, the attack surface increases while visibility decreases, leading to new security concerns.

Observability provides DBAs with a better understanding of how developer teams use data so they can more easily ensure the data remains secure and protected. Additionally, many modern IT environments deploy threat intelligence solutions to observe actions and detect threat signals to help predict security incidents and respond before they happen. Some threat intelligence applications are powered by AI, which can automatically gather and organise complex log data from the entire network. This data is then compared with an out-of-the-box threat database, helping to detect suspicious activity early. Observability and threat intelligence working in tangent provide a comprehensive overview to help relieve some security concerns for DBAs and create a more secure environment for data to be shared between teams.

Mastering data compliance

Ensuring data is compliant is top of mind for every organisation. Compliance regulations such as GDPR, ACCC and ASIC carry severe penalties for data leaks. Companies must maintain the integrity and availability of sensitive and regulated data.

Ensuring compliance while managing requests for information from developers working across geographies and clouds can be a daunting task for DBAs. They must not only remain compliant, but many businesses must demonstrate continuous compliance, which is only possible using complex, dynamic and automated systems.

Observability allows teams to clearly see and follow the complex data flow as it moves between DBAs, developers and other teams. Additionally, observability can be deployed to help ensure that any sensitive data required to be anonymised is anonymised before it is shared. DBAs can also leverage observability to track the lineage of data as it moves between teams to ensure it is not changed or transformed in any way risking a compliance violation. Making compliance a regular part of the IT environment should not be viewed as a burden as it provides many benefits, including improved data quality, which can be used to innovate further and build better products.

Enhancing team productivity

Observability can support harmony between DBAs and developers by helping mitigate security and compliance concerns. Not only is it addressing these issues, but it is being used to support increased productivity amongst teams.

For DBAs, this means gaining additional insights to help ensure applications and services run properly. Observability collects information from hybrid and multi-cloud environments and presents it in a single view, allowing DBAs to respond quickly and resolve issues. With this insight, DBAs can better manage critical data and anticipate future problems before they arise.

Observability plays a crucial role in enhancing developers’ productivity when building applications. The ‘shift left’ trend has made observability indispensable, enabling a comprehensive view of the build process through a single-pane-of-glass interface. This allows developers to gain valuable insights into the performance of web applications and promptly address any issues that arise. Additionally, observability facilitates faster innovation by providing live code profiling, enabling the prevention of future problems.

Observability offers a tangible solution by ensuring that data is not subject to the tragedy of commons, where its misuse can negatively impact an entire organisation. It empowers teams to securely leverage data, thereby increasing productivity and fostering the development of innovative applications.

Image credit: iStock.com/simoncarter