Introduction
Climate change is a topic that is currently one of the most discussed and debated issues globally. It is a phenomenon that is closely linked to human activities, and its effects are far-reaching and serious. Climate change is characterized by the continuous increase in temperatures caused by greenhouse gases emission. The primary cause of greenhouse gases emission is human activities such as burning fossil fuels, deforestation, and agriculture. These anthropogenic activities lead to an increase in CO2, CH4, and other greenhouse gases that trap heat in the earth's atmosphere, resulting in global warming.
To combat climate change, we must reduce greenhouse gases emissions, plant trees, and support policies that promote renewable energy. The fight against climate change is a marathon, and every step counts.
Who emits more carbon?
According to International energy agency (IEA)
1. Industries - 18%
2. Transport - cars, airplanes - 25%
3. Electricity & heat producers - 42%
So now the question is, where do all the carbon emission come from, in the software world?
The current estimation is that, the data centers and data transmissions consume large amounts of energy.
The applications that we develop are deployed in servers and these servers are hosted in data centers. Due to the high demand of digital services, the data center markets are also seeing rapid growth.
For example, data center electricity use in Ireland has more than tripled since 2015, accounting for 14% of total electricity consumption in 2021.
In Denmark, data center energy use is thought to be tripled by 2025, this means it will account for around 7% of the country’s electricity use.
The energy consumption and subsequent carbon emissions from data centers can be attributed to many factors such as:
Powering the servers and cooling systems:
Servers generate an immense amount of heat, hence require cooling systems which can consume a significant amount of electricity.
Redundancy and back-up measures:
Data centers often have back-up systems in place to ensure that services are not disrupted in case of a power outage or system failure. These systems need to be powered and hence consume additional energy.
Growth in demand for digital services:
With more and more people relying on digital services such as online shopping, entertainment, and social media, the demand for data centers and subsequently the energy consumption is on the rise.
Renewal energy:
Efforts have been made to reduce the carbon footprint of data centers, such as the use of renewable energy sources to power the centers like solar or wind. Data centers can also employ energy-efficient measures such as using virtualization techniques to consolidate servers, upgrading to more energy-efficient hardware, and employing temperature control systems that respond to server demand.
Data Transmission
Data transmission refers to the process of sending and receiving data over a network, which could be the internet or any other communication network. When users interact with applications or websites, data is often transmitted between the user's device and the application's server, or between different servers. The data could be in the form of text, images, audio, video, or any other type of digital information. The transmission of data is typically facilitated by various network protocols and technologies, such as TCP/IP, HTTP, FTP, and others. Secure data transmission is also an important consideration, which is often achieved through encryption and other security measures.
According to a report by the International Energy Agency (IEA) published in 2020, that data centers and data transmissions consume significant amounts of electricity in software industry, this was estimated to consume about 1% of global electricity consumption in 2019.
As software engineers, the software that we build, are used around globally. So, is there anything that we can contribute to fight for the climate change. There are huge opportunities to help the climate change crisis
How to minimize the carbon emission in data transmission?
We can build software that is optimized for energy efficiency, such as minimizing the amount of data that needs to be transmitted over networks, reducing unnecessary computations, and ensuring that software is properly optimized for the hardware it runs on.
By minimize the distance that data needs to travel. This can be achieved by placing servers closer to the end-users or consumers of the data. This approach is sometimes referred to as "edge computing" or "fog computing," where computing resources are distributed at the edge of the network, closer to where the data is generated or consumed. By bringing computing resources closer to the end-users, data can be processed locally, which reduces the amount of data that needs to be transmitted over long distances.
Edge computing can also reduce latency, which is the time it takes for data to travel between the end-user and the server. This is especially important for applications that require real-time responses, such as gaming or video streaming.
Minimise the amount of data that needs to be transmitted. This can be achieved by filtering out unnecessary data and transmitting only the data that is required.There are several techniques that can be used to reduce the size of data transmitted over networks:Compression: Data can be compressed before transmission to reduce its size. Compression algorithms can be applied to text, images, audio, and video data to reduce the amount of data that needs to be transmitted.
Caching: Data that is frequently accessed can be cached on the client side or at intermediate points in the network. This reduces the amount of data that needs to be transmitted over the network.
Filtering: Data can be filtered to remove unnecessary or redundant data before transmission. For example, in a video stream, frames that are very similar to each other can be filtered out to reduce the amount of data that needs to be transmitted.
Consider the data center regions which consumes green energy. Green energy means, energy produced by solar or wind. This will be zero or low carbon emission. One way to reduce the carbon footprint of data centers is to power them with renewable energy sources such as solar or wind power. By using renewable energy, data centers can significantly reduce their carbon emissions, as these sources of energy produce very little or no carbon emissions.
There are several ways to power data centers with renewable energy:
On-site renewable energy: Data centers can install solar panels or wind turbines on-site to generate renewable energy. This can help reduce the carbon emissions associated with energy consumption in the data center.
Off-site renewable energy: Data centers can purchase renewable energy from off-site sources, such as wind or solar farms. Many utilities now offer renewable energy purchasing programs that allow data centers to purchase renewable energy credits or sign power purchase agreements to procure renewable energy.
Renewable energy certificates: Data centers can purchase renewable energy certificates (RECs) to support renewable energy production. RECs represent the environmental benefits associated with renewable energy production and can be purchased to offset the carbon emissions associated with energy consumption in the data center.
Consider deploying the batch jobs which are mostly not consumer interacted applications in data centers which uses green electricity and even if its far from the consumer’s location, that’s not an issue. Batch jobs are often non-interactive applications that can be scheduled to run at specific times or intervals, without the need for real-time interaction with end-users. Because batch jobs do not require real-time interaction, their physical location is less important than the location of interactive applications that require low latency.
Deploying batch jobs in data centers powered by renewable energy sources can help reduce the carbon emissions associated with their operation.
Furthermore, renewable energy sources are often less expensive than traditional sources of energy in the long term, making them a cost-effective option for powering data centers.
However, it's important to note that even though batch jobs do not require real-time interaction with end-users, their output or results may still be important to end-users. Therefore, it's important to ensure that the output or results of batch jobs are transmitted to end-users in a timely and efficient manner, even if the data center is located far from them.
Moving from on-premises data centers to public cloud infrastructure can significantly reduce carbon emissions.
If we consider public cloud instead of on-premises, it will immediately reduce 84% of carbon emissions as a result. This reduction results from a combination of factors, including the fact that cloud data centers use less physical space and resources than those of on-premises; the use of more efficient servers and higher server usage; and the fact that public cloud providers are themselves committed to sustainable infrastructure and reduced carbon footprint, which will enhance and support their customer’s efforts as well.
There are several reasons why this is the case:
Efficiency: Public cloud providers typically use more efficient servers and data center infrastructure than on-premises data centers. Cloud providers also have economies of scale that allow them to optimize resource usage and reduce waste.
Renewable energy: Many public cloud providers have made commitments to power their data centers with renewable energy sources, such as wind or solar power. By using renewable energy sources, cloud providers can significantly reduce their carbon emissions.
Location: Public cloud providers typically locate their data centers in areas with access to renewable energy sources and cool climates that reduce the need for energy-intensive cooling systems.
Multi-tenancy: Cloud infrastructure allows multiple customers to share computing resources, which can lead to higher server utilization rates and less waste.
By leveraging the efficiency, renewable energy sources, and location advantages of public cloud infrastructure, organizations can significantly reduce their carbon emissions compared to on-premises data centers. Additionally, many cloud providers are committed to sustainability and reducing their own carbon footprint, which can further enhance and support their customers' efforts to reduce carbon emissions
Idle Servers
Up to 50% of the servers in data centre are idle and not doing anything, they are just waste of energy. Most of the time, they don’t do anything but just wait for the active server to fail, so that they can take over when its fails. And while its waiting it will consumes electricity and emits carbon.
Many data centers have a significant portion of servers that are not being used at any given time, often referred to as "zombie servers." These servers consume electricity and emit carbon even though they are not actively performing any useful work.
One reason for this is the practice of using redundant servers for failover purposes. Redundant servers are kept on standby, ready to take over in the event of a failure of the active server. However, these servers are often idle for long periods of time and consume energy unnecessarily.
To address this issue, data centers can implement more efficient failover mechanisms that use virtualization technologies, which allow for more efficient resource utilization. For example, virtual machines can be dynamically moved between physical servers as needed, allowing for better use of computing resources and reducing the number of idle servers.
Another approach is to implement energy-efficient hardware, such as processors with low-power modes or hardware accelerators that can perform specific tasks more efficiently. This can help to reduce energy consumption across the entire data center, including idle servers.
Reducing the number of idle servers in data centers is an important step in reducing energy consumption and carbon emissions. By implementing more efficient failover mechanisms and using energy-efficient hardware, data centers can reduce waste and operate more sustainably
Instead, we can use containers which is a high availability model, introduced in cloud technology, where we do not need any redundant servers. When there is a failure, cloud can start a new container. Starting a new container is much quicker and containers consume less electricity and less resources compare to a physical server.
Containers are a great solution for improving server efficiency and reducing energy consumption in data centers. Unlike physical servers, containers can be created and destroyed quickly and easily, making them a great fit for cloud-based, high-availability applications.
Containers can help reduce the number of idle servers in data centers because they can be scaled up or down based on demand. This means that resources are allocated only when needed, rather than having idle servers consuming energy and resources. Additionally, containers can be migrated between physical hosts without interrupting the application, further increasing their efficiency.
Furthermore, containerization can also help to reduce the overall energy consumption of data centers by reducing the need for hardware upgrades. Because containers can be deployed on existing hardware, it is often possible to extend the useful life of servers and other infrastructure, reducing the need for new hardware and the associated energy consumption.
Containerization is an important tool for improving the efficiency and sustainability of data centers. By reducing the number of idle servers and optimizing resource utilization, containers can help to reduce energy consumption and carbon emissions in data centers
Lesser used Services
Applications that are idle for most of the time are another example of wasted energy in data centers. These applications can consume energy and resources even when they are not performing any useful work, leading to unnecessary carbon emissions.
To address this issue, there are a few different strategies that can be used. One approach is to implement auto-scaling policies that dynamically adjust the resources allocated to the application based on demand. This can help to ensure that the application has enough resources to handle incoming requests, while also minimizing waste when demand is low.
Another approach is to use serverless computing technologies, which allow applications to run only when triggered by an event or request. In a serverless architecture, resources are allocated only when needed, and are released when the task is completed. This can help to reduce energy consumption and waste, particularly for applications that are not frequently used.
Overall, reducing the energy consumption of applications that are idle for most of the time is an important step in improving the sustainability of data centers. By implementing auto-scaling policies or serverless architectures, data center operators can optimize resource utilization, reduce waste, and minimize carbon emissions.
• HTTP requests,
• Events,
• New messages in queue,
• Task scheduler
Once, the functionality is done it will terminate and it does not use CPU or memory or wastes energy.
• AWS Lambda
• Azure Functions
• Google Cloud Functions
Architecture
The most microservices architecture which I have worked with, have a business layer and CRUD layer. In some cases, the business and crud services are one to one relationship.
Microservices architectures can result in a large number of components, which can be time-consuming and resource-intensive to develop and maintain. In some cases, the benefits of a microservices architecture may not outweigh the costs, particularly if the system is relatively simple and does not require a high degree of scalability or flexibility.
However, there are some cases where a microservices architecture can be a good fit. For example, if the system needs to be highly scalable, or if different components have different performance or availability requirements, a microservices architecture can help to provide the necessary flexibility and modularity.
To help mitigate the complexity and maintenance overhead of a microservices architecture, it is important to carefully design and architect the system, and to use appropriate tools and technologies to support development, testing, and deployment. This may include using automated testing and deployment pipelines, containerization technologies like Docker, and orchestration tools like Kubernetes or Docker Swarm.
Overall, while microservices architectures can be complex and resource-intensive to develop and maintain, they can also provide significant benefits in terms of scalability, flexibility, and modularity.
Here with I’m proposing a hybrid architecture that combines elements of both monolithic and microservices architectures. By combining the business and CRUD layers which have one to one relationship into a single service, we can potentially reduce the number of components and network traffic between them.
This approach can have several potential benefits, including reduced development and maintenance time, as well as potentially lower infrastructure costs. By reducing the number of components, we may also be able to simplify the testing and deployment process.
Monolithic Architecture
Monolithic architecture is not always a bad choice and can be a good fit for smaller applications with less complexity. In some cases, the overhead of deploying and maintaining a microservices architecture may outweigh the benefits.
In a monolithic architecture, all components of the application are bundled together into a single deployable unit. This can simplify development, testing, and deployment processes, especially for smaller applications that don't require the scalability and flexibility of a microservices architecture.
When running batch jobs that require a large amount of resources, it's better not to use shared microservices. Using shared microservices can result in contention for resources, which can lead to slower job execution and longer running times. It can also lead to a waste of energy and resources, which is not ideal for reducing carbon emissions. Instead, it's better to provision dedicated resources for batch jobs that require a lot of processing power. This ensures that there are enough resources available to complete the job quickly and efficiently, without impacting other applications running on shared resources. This approach can also help reduce energy consumption by avoiding the unnecessary use of shared resources
Monolithic architecture for batch jobs which connects to the database
Microservice architecture for batch jobs with dedicated microservice
Batch deployment in dedicate cluster
Energy efficient databases
There are many databases that are designed to be energy-efficient, with different approaches to achieving this goal. Here are some examples: PostgreSQL: PostgreSQL is an open-source relational database that includes features to help reduce energy consumption. It offers a variety of performance tuning settings that can be adjusted to optimize energy usage. MySQL: MySQL is a popular open-source database that has been designed with energy efficiency in mind. It includes features such as query caching and thread pooling to help reduce energy consumption. MongoDB: MongoDB is a NoSQL database that is designed to be highly scalable and energy-efficient. It uses a distributed architecture and allows users to configure replica sets and sharding to optimize energy usage. InfluxDB: InfluxDB is a time-series database that is designed for storing and querying large amounts of data with high write throughput. It includes features such as data compression and indexing to help reduce energy consumption. Apache Cassandra: Apache Cassandra is a distributed NoSQL database that is designed to be highly scalable and energy-efficient. It includes features such as data replication and load balancing to help optimize energy usage. Microsoft SQL Server: Microsoft SQL Server is a popular relational database that includes features such as resource governor and dynamic management views to help reduce energy consumption. Oracle Database: Oracle Database is an enterprise-level relational database that includes features such as automatic storage management and dynamic resource allocation to help optimize energy usage.
These are just a few examples of databases that are designed to be energy-efficient. When choosing a database, it's important to consider not only its energy efficiency but also its other features and capabilities, as well as its compatibility with your application and infrastructure.
Data
Reduce data storage and usage to an absolute minimum. We can reach this goal by implementing a data classification policy.
Define which data to store, how long, and why.
Define fast and slow access classes and storage types.
Start by understanding your data's impact on consumed resources and delete unnecessary data while using data life-cycle policies.
Minimize over provisioning storage and prevent data movement across network boundaries as much as possible.
Use data-access technologies that support location-aware data retrieval and different storage patterns.
Use compression and deduplication technologies to reduce the storage footprint of data.
Implement data archiving policies to move infrequently accessed data to cheaper storage tiers or even offline storage.
Regularly perform data audits to identify and delete stale, outdated or redundant data.
Use encryption and access control mechanisms to ensure that only authorized personnel have access to sensitive data.
Consider using cloud storage services that allow you to scale storage up and down as needed, so you only pay for what you use.
Use data caching technologies to reduce the need to retrieve data from storage every time it is accessed.
Monitor data usage and performance metrics to identify opportunities to optimize data storage and retrieval.
Logs
Sending and storing logs from all possible sources (workloads, services, diagnostics and platform activity) can considerably increase storage and network traffic, which would impact higher costs and carbon emissions.
It's important to carefully consider the log data you're collecting and storing, and ensure that it's necessary for your business or operational requirements. Here are some best practices for managing logs efficiently:
Determine the log data you need: Identify the types of log data that are essential for your business or operational needs and determine the retention period for each. This will help you avoid storing unnecessary data and minimize storage costs.
Set up log rotation: Configure your log collection system to rotate logs on a regular basis, such as daily or weekly. This will prevent logs from becoming too large and consuming excessive storage.
Use log compression: Compressing logs before storing them can significantly reduce their size, resulting in lower storage costs and reduced network traffic.
Filter logs: Use filters to exclude unnecessary logs and focus on the ones that are important for your needs. This can also help reduce network traffic and storage costs.
Utilize log aggregation: Instead of storing logs in separate locations, consider using a log aggregation system that centralizes all your logs in one place. This can make it easier to manage and analyze logs, and reduce the amount of storage and network traffic required.
By implementing these practices, you can minimize the storage and network impact of log data, reduce costs, and lower carbon emissions
Programming Languages
The compiled languages proved the fastest and most energy-efficient
According to the research, the leading list in energy efficient languages are C, Rust, C++, Ada. Java was ranked in next. Java is considered to be an energy-efficient language. While compiled languages like C, Rust, and C++ may be more energy-efficient in some cases, Java's efficient memory management and garbage collection features can make it a more sustainable option in certain scenarios. Additionally, Java's popularity and extensive tooling support can make it a practical choice for developers and organizations seeking to build sustainable software.
Normalised global results for energy, time and memory
Green coding
Writing efficient code: This involves optimizing code to reduce energy consumption, such as reducing memory usage, minimizing network traffic, and avoiding unnecessary processing.
Choosing efficient programming languages and frameworks: Certain programming languages and frameworks are more energy-efficient than others. For example, compiled languages like C and Rust are typically more efficient than interpreted languages like Python.
Using cloud-based infrastructure: Cloud providers can help reduce the carbon footprint of software development by providing energy-efficient data centers and by allowing for more efficient resource utilization.
Adopting sustainable development practices: This includes using agile development methodologies, reducing waste, and minimizing unnecessary travel.
Green coding is a relatively new practice that minimize the energy in processing lines of code.
This is written in a way that minimize energy consumption of software. Software applications and programs are instructed by lines of code written by engineers. As each line of code is parsed, the device, processes this code, which rise to carbon emissions. The more code to process, the higher level of emissions.
Approximately 90% of software developments leverages open-source code which is not tailored for specific uses and applications. It therefore contains redundant code and more lines of codes. This will be using more processing power and giving rise to unnecessary carbon emissions
Best ways to cut the carbon footprint of your cloud usage
Resizing instances according to demand:
Resizing instances according to demand is another way to reduce carbon footprint in the cloud. When an instance is underutilized, it still consumes energy and emits carbon, even if it is not performing any useful work. Resizing instances to match the current demand can reduce energy consumption and carbon emissions.
Cloud providers like AWS, Azure, and Google Cloud offer autoscaling features that can automatically resize instances based on demand. By using these features, you can ensure that your instances are not over-provisioned or underutilized, reducing unnecessary energy consumption and carbon emissions. Additionally, you can manually adjust the instance sizes based on your workload demand patterns to optimize your resource utilization and minimize your carbon footprint.
Switch server instances off:
One way to reduce the carbon footprint of cloud usage is to switch server instances off when they are not being used. This can be done by implementing auto-scaling policies that automatically spin up new instances when demand is high, and then shut them down when demand decreases. By doing this, you can ensure that you are only using the resources that you need, and not wasting energy on idle servers.
Optimize resource usage:
Another way to reduce the carbon footprint of your cloud usage is to optimize resource usage. This can be done by using tools that monitor resource usage and identify areas where resources are being wasted. For example, you may find that certain instances are using more CPU or memory than they need to, or that certain applications are generating excessive network traffic. By identifying these issues, you can take steps to optimize your usage and reduce your energy consumption.
Renewable Energy:
Choose a cloud provider with renewable energy commitments: Look for cloud providers that have made commitments to use renewable energy sources to power their data centers. This will ensure that the energy used to power your cloud services comes from renewable sources and reduces the carbon footprint of your cloud usage.
Use cloud services efficiently:
Optimize your cloud services to ensure that you are only using what you need. This means choosing the right-sized instances and storage options for your workload, using auto-scaling to ensure that resources are only used when needed, and turning off unused resources to reduce waste.
Reduce data transfers:
Data transfers in the cloud can consume a lot of energy and contribute to your carbon footprint. You can reduce data transfers by using edge computing, which processes data closer to the source, or by using caching and content delivery networks (CDNs) to store and serve content locally. This reduces the need for data transfers and can significantly cut your carbon footprint
Planning cloud usage in advance
Planning your cloud usage in advance can help reduce emissions in a few ways. First, it allows you to optimize resource usage, which in turn reduces the amount of energy needed to power and cool the servers. This means that you can accomplish the same amount of work with fewer resources, which ultimately results in lower emissions.
Secondly, planning ahead can help you avoid spikes in usage that might require additional resources to be spun up quickly. This sudden increase in demand can lead to servers being provisioned inefficiently, which can result in higher emissions. By planning ahead and anticipating spikes in usage, you can make sure that the necessary resources are provisioned in a more efficient and sustainable manner.
Finally, planning ahead can help you take advantage of cloud providers that have committed to using renewable energy sources. By choosing providers that are powered by clean energy, you can further reduce the carbon footprint of your cloud usage
AWS & Azure have both pledged to use 100% renewable energy by 2025
By committing to 100% renewable energy usage, both AWS and Azure are demonstrating their commitment to sustainability and reducing their carbon footprint. This is an important step in addressing the climate change crisis, as the digital economy continues to grow and consume more energy
Google has already declared itself carbon-neutral and pledges to run on 100% carbon-free energy by 2030
Google has been carbon-neutral since 2007, meaning that the company has balanced its carbon emissions with equivalent carbon offsets or renewable energy purchases. In 2020, Google announced a new goal to run its data centers and campuses on 100% carbon-free energy by 2030, which includes both electricity and thermal energy.By committing to run its operations on 100% carbon-free energy by 2030, Google is taking a significant step in addressing the climate change crisis. As one of the world's largest technology companies, Google's actions can have a significant impact on reducing carbon emissions and promoting the transition to renewable energy sources
Conclusion
In conclusion, the software industry has a significant role to play in reducing carbon emissions and mitigating the impact of climate change. There are several ways to achieve this, including optimizing code, reducing data storage, using energy-efficient languages, and adopting cloud computing best practices.
By adopting these practices, software companies can reduce their carbon footprint, save energy, and contribute to a more sustainable future. It is important for companies to recognize the impact of their operations on the environment and take proactive steps to reduce their emissions.
Moreover, the software industry has an opportunity to innovate and create new solutions that can help other industries reduce their carbon emissions. By leveraging the power of technology, software engineers can develop innovative solutions that promote sustainability and help combat climate change.
.png)
