Distributed computing is about many separate computers working together to solve a big problem or share resources, acting like one powerful system.

Key idea: Distributed systems are defined by independent computers with local memory that communicate through message passing, presenting challenges in coordination and fault tolerance.

So, what makes a system 'distributed'? It is not just about having many computers. The key is that these computers are separate and independent, each with its own memory. They cannot just peek into each other's thoughts; they have to explicitly send messages to share information.

This setup brings some interesting challenges. For example, how do you make sure everyone is working on the right part of the puzzle at the same time, without stepping on each other's toes? How do you coordinate actions when there is no single master clock telling everyone what time it is? And what happens if one computer suddenly stops working? A good distributed system is designed to handle these issues, making it robust and reliable.

The term 'distributed' used to mean computers spread out geographically, like across different cities. But today, it can even mean different programs running on the same physical computer, as long as they act independently and communicate by sending messages.

Key idea: Events notify interested parties about something that happened, while messages are a broader form of communication that can include commands or data for specific recipients.

When these independent computers need to talk, they do so using messages. Think of it like sending notes or emails between team members. But there are different kinds of 'notes' they can send.

An event is like a notification that something has happened, for example, 'Order Placed'. It is usually broadcast to anyone who might be interested, and the sender does not typically wait for an immediate reply. This makes systems very flexible because the sender does not need to know who will receive the event or what they will do with it.

A message is a broader term. It can be an event, but it can also be a direct command, like 'Process Payment', or just a document with data. Messages often imply a more targeted communication, where one computer sends something specifically to another, sometimes expecting a response. Modern systems often use a mix of both: events to announce changes and messages for specific actions or workflows.

⚠️Watch out: People often confuse events and messages, but the key difference is that events are typically broadcast notifications of a state change, while messages can be more targeted commands or data transfers.

Key idea: Parallel computing involves multiple processors sharing a single memory, while distributed computing uses separate computers with private memories that communicate via messages.

You might hear about 'parallel computing' and 'distributed computing' and wonder if they are the same. They are very similar, and often overlap, but there is a key difference in how the computers share information.

In parallel computing, imagine a group of workers all looking at the same whiteboard to get their instructions and share their progress. They all have access to a shared memory space. This is like multiple processors inside a single computer, all directly accessing the same pool of data.

In distributed computing, as we have discussed, each computer has its own private memory. They cannot directly see each other's data. Instead, they have to explicitly send messages to exchange information. So, parallel computing is like a tightly knit team sharing one big workspace, while distributed computing is like a team of people in separate offices sending emails to each other.

Why go through all this trouble of connecting many computers? There are several powerful reasons. Sometimes, the task itself demands it, like when data is created in one place but needed in another, such as a global banking system.

Even when one supercomputer could theoretically do the job, using a distributed system often makes more practical sense. It can give you much more storage, faster processing, and higher bandwidth than any single machine could offer. Think of how much data Google processes or how many users Facebook serves; no single computer could handle that.

Crucially, distributed systems are often more reliable. If one computer fails, the others can keep working, preventing the entire system from crashing. This is called fault tolerance. They are also easier to expand; you just add more computers as needed. And surprisingly, sometimes it is cheaper to buy many less powerful computers than one extremely powerful one to get the same overall performance.

“A well designed distributed system is more scalable, more durable, more changeable, and more fine tuned than a monolithic application deployed on a single machine.”

Key idea: Distributed computing evolved from early operating systems and networks like ARPANET, becoming a distinct field of study by the 1980s with email as an early success.

The idea of computers working together by sending messages has been around for a long time. It started in the 1960s with early operating systems. In the 1970s, local area networks like Ethernet emerged, allowing computers in the same building to talk to each other.

One of the earliest and most successful examples of a large scale distributed application was email, which became popular on ARPANET (a precursor to the internet) in the early 1970s. This showed the power of connecting computers across distances to share information.

By the late 1970s and early 1980s, distributed computing became its own dedicated field of study in computer science, with conferences specifically focused on its principles and challenges. This marked its recognition as a fundamental and complex area of research.

“Email became the most successful application of ARPANET, and it is probably the earliest example of a large scale distributed application.”

Key idea: Coordinator election is the process by which computers in a distributed system agree on a single leader to organize tasks, often by comparing unique identifiers.

A common challenge in distributed systems is how to pick one computer to be the 'leader' or 'coordinator' for a specific task. Imagine a group of people who all need to decide who will lead a project, but they cannot all talk at once. They need a fair way to choose.

This is called coordinator election. Before the election, no one knows who the leader is, or maybe the old leader is gone. After the election, every computer in the network agrees on which single computer is the new coordinator. This coordinator then helps organize the work among the others.

Computers often use unique IDs to break ties. For example, they might all compare their ID numbers, and the one with the highest number wins the election. These algorithms are designed to be efficient, using as few messages and as little time as possible to make the decision.

Key idea: Distributed computing is fundamental to many modern technologies, enabling internet services, online gaming, scientific research, and critical infrastructure that require high reliability and scalability.

Distributed computing is everywhere in our daily lives, even if we do not always see it. When you browse the internet, you are using a massive distributed system. When you play an online game with friends, or check your bank balance, or even use a navigation app, you are interacting with distributed systems.

They are essential for things that need huge amounts of processing power, like scientific research, or for systems that absolutely cannot fail, like aircraft control. They also power things that need to scale up and down quickly, like cloud services.

From telecommunications networks that connect our phones, to the World Wide Web itself, to the complex systems that manage airline reservations, distributed computing makes modern technology possible by allowing many machines to work together seamlessly.