Franco Fernando

Consistent hashing and rendezvous hashing explained.

2021-12-24T00:00:00+00:00

If you are familiar with the hash table data structure you are certainly familiar with the concept of hashing. Hashing is the process of transforming an arbitrary piece of data into a fixed size value (usually an integer).

But where and why should you use hashing in a distributed system?

A first scenario is when you need to provide elastic scaling (a term used to describe dynamic addition or removal of servers based on usage load) for cache servers.

A second scenario is when you need to scale out a set of storage nodes like for example NoSQL databases.

Let's check both scenarios in details with a couple of examples.

Scaling cache servers

Let's suppose we have a system composed by multiple clients talking with multiple servers through a load balancer. So, the clients requests go to the load balancer that reroutes them to servers. The load balancer shall have a server selection strategy, something to pick each time the server to which redirect the requests.

If the requests of the clients are computationally very expensive it is reasonable using a fast in memory cache on each server. If the response to a client request is in the cache, the server can immediately return the response.

To make an effective use of caching the server selection strategy shall guarantee that all the same kind of requests will be rerouted to the same server.

Here hashing comes into play. We can hash the requests coming to the load balancer and use that hash to reroute the requests according to the hash value. For example we could hash the IP address of the client, its username or the HTTP request.

This solves the problem of mapping request to same server, but what happen when cache servers are dynamically added or removed?

Partition data across multiple servers

Let's suppose that we have a web application and you want to shard your data across multiple database servers.

There are two challenges we need to face while designing our database backend:

how do we know on which server a particular piece of data will be stored?
when we dynamically add/remove a database server, how do we know which data will be moved from the existing servers to the new ones? And how do we minimize data movement between servers in that case?

Once again hashing can help in this scenario. We can hash the data key and use that hash to find the database server where the data is stored. This solves the first challenge, but what happen when database servers are dynamically added or removed?

Problem Generalization

The problems described in the previous scenarios can be easily generalized. We have 3 entities

Keys: unique identifiers for data or workload requests
Values: data or workload requests that consume resources
Servers: entities that manage data or workload requests

The goal is to map keys to servers while preserving three properties:

Load Balancing: each server should be responsible for approximately the same data or load.
Scalability: servers should be added or removed with low computational effort.
Lookup Speed: given a key, it should be possible to efficiently identify the correct server.

Naive approach

Let's start with a simple way we can use to assign the clients requests or the data keys to the servers considering their hash value. Let's suppose we have N servers. The simplest strategy is to

number the servers from 0 to N-1;
calculate the hash value of the key modulo N to find a server number;
assign the key to the server identified by that number.

This schema assigns the key always to the same server, but has a main problem: it is not horizontally scalable. If we add or remove servers from the set keeping N as reference number, the requests will continue to be routed to the removed one or will never go to the new one. If we module against the new number of servers instead, all the existing mappings get broken with the result that all existing data needs to be remapped and migrated to different servers.

This would be expensive and uncomfortable because it will either require a scheduled system downtime to update the mappings or creating read replicas to serve the queries during the migration. We need a better approach.

Consistent hashing

Let's suppose that the space of possible hash values is the range of 32 bit unsigned integers [2^32-1,0] and let's image that this space is organized in a circular way such that the last value wraps around. The consistent hashing approach requires the following steps:

Place the servers in the circular space. We use an hash function to put the initial list of servers into specific places on the circular space. For example, we can hash IP addresses of the servers to map them to different integers in the circle.
Place the keys in the circular space. We use an hash function to map each key to an integer in the circle.
Assign keys to servers. If the hash value of the key maps directly onto the same place of a server, we assign the key to that server. Otherwise, we assign the key to the first server we meet moving clockwise from the place where the key is mapped.

This schema works great when we need to horizontally scale our system. Indeed when a server is added or removed from the ring, only the next server on the circle is affected.

If we add another server we don't need to remap all the keys, but only the keys that are placed between the previous server in the ring and the new one. Assuming an uniform distribution of the keys on the circle, we need to remap only k/n keys on an average where k is the number of keys and n is the number of servers.

If we remove a server instead, the next server on the circle becomes responsible for all of the keys stored on the that server.

However, this scheme can still generate a non uniform load distribution between the servers.

Even if the servers were initially evenly distributed across the circle, after adding and removing some servers we can run into a situation where the server distribution across the circle is non anymore balanced. Some of the servers become responsible for a larger portion of the keys.

In reality, the issue is even more complicated because the keys don't have a perfectly uniform distribution in most cases. The non uniform distribution of the keys coupled together with non uniform distribution of the servers on the circle can lead to a situation where some servers get more easily overloaded becoming hotspots and degrading the performance of the whole system.

Virtual Nodes

The aforementioned problem can be solved introducing a number of replicas or virtual nodes for each server across the circle. This means that we use multiple hash functions to map each server into multiple places on the circular space.

The virtual nodes divide the circular space into multiple smaller ranges and each physical server is assigned to several of these smaller ranges. Virtual nodes are randomly distributed across the circle and are generally non-contiguous so that no neighboring nodes are assigned to the same physical server.

As the number of replicas or virtual nodes in the circle increase, the key distribution becomes more and more uniform. In real systems, the number of virtual nodes is very large.

Virtual nodes not only help to avoid hotspots in the system, but they bring also the following advantages:

virtual nodes divide the hash circular space into smaller subranges, speeding up the rebalancing process after adding or removing nodes because the keys to be reassigned involves multiple physical servers instead of one.
virtual nodes can carry physical replicas of a server for fault tolerance
virtual nodes make it easier to maintain a cluster of servers made by heterogeneous machines, because it is possible to assign a higher number of virtual nodes (and hence circle subranges) to powerful servers and a lower number to less powerful servers.

Consistent Hashing in Action in Production Systems

There are a number of live systems which use consistent hashing including:

Couchbase
Apache Cassandra
Amazon Dynamo
Riak

for data partitioning and

Akamai Content Delivery Network
Discord Chat

for caching purposes.

Rendezvouz Hashing

Rendezvous hashing is less known hashing strategy, providing a different approach respect to consistent hashing.

The basic idea is to calculate a ranking of all the possible servers for each key using some logic. Each key is assigned to the server with highest ranking for that key.

The advantage of this strategy is clear. When adding or removing a server S, only the keys for which the S has highest ranking are affected. When removing S, the keys are assigned to the server with second highest ranking in their list. When adding S, only the keys for which S has the highest ranking are assigned to S.

Rendezvous hashing is much simpler to implement than consistent hashing in practice. You can easily get a unique list of servers for each key by hashing the key together with the server and sort the servers based on the hash values. Some simple way hashing keys and servers togheter are concatenating the key with the server or using the server ID as a hash seed.

To summarize, the rendezvous hashing approach requires three steps:

hash all the possible key-server combinations with a hash function
assign each key to the server with the highest hash value
maintain the association between each key and the server with the highest hash value after adding and removing servers

Comparison between Rendezvouz and Consistent Hashing

There are 3 main things to consider when comparing rendezvouz and consistent hashing.

Query time. Consistent hashing uses typically binary search to map a key to the closest server in the circular space. So each query takes O(log(NV)) time, where N is the number of servers and V the number of virtual nodes for each server. Rendezvouz hashing takes typically O(N) because we have to calculate the hash for each key-server combination.
Memory requirements. Consistent hashing requires some fixed memory to work well in order to store the hash values for server and virtual nodes plus the mapping between servers and virtual nodes. Rendezvous hashing doesn't require storing any additional data.
Complexity. Rendezvouz hashing is easier to explain, understand and implement than consistent hashing. It provides an even distribution of the keys when adding and removing, assuming a good choice of the hash function. Consistent hashing alone (withou virtual nodes) fails to provide an even distribution of the keys especially for small clusters..

In summary, consistent hashing trades load balancing for scalability and lookup speed while rendezvous hashing provides an alternative tradeoff that emphasizes equal load balancing. Consistent hashing is more extensively used, but rendezvous hashing can be a good algorithm to load balance medium-size distributed systems, where an O(N) lookup cost is not prohibitive.

Conclusion

Consistent hashing and rendezvous hashing are great techniques used in countless distributed systems. They come in handy in many real world scenarios from elastically load balancing servers to sharding data between servers. I hope this post helped you in understanding better how these techniques are implemented and how you can use them while designing a distributed system.

If you liked this post, follow me on Twitter to get more related content daily!

The definitive guide to load balancers.

2021-11-17T00:00:00+00:00

Load balancers are an extremely important topic in distributed systems. They are used in almost every large scale system. Keep reading to know what they are and understand how they works.

Introduction

Let's start with a simple use case where we have a client issuing requests to server. The server performs some business logic, fetch some data from a database and then returns responses to the client.

All works fine, but what happens if the system becomes bigger?

Let's say we have now N clients issuing multiple requests. The server has limited resources and throughput.The more requests the server receives the more it get overloaded. Our system becomes soon very slow or even worst a failure happens.

How can we prevent this?

Of course we can scaling our system. There are 2 possibilities:

vertical scaling, increasing the power of the server;
horizontal scaling, adding more servers.

The problem of vertical scaling is that it is limited by the maximum amount of resources a single server can have. At a certain point we need to scale horizontally.

Teoretically, if we add M servers having the same resources, the system can handle M times the initial load. But this is true only if the requests of the client are equally distributed across the servers. Here is exactly where the load balancer comes into play.

Definition

A load balancer is a server sitting between a set of clients and a set of servers and performing the job of balancing the clients requests between the servers. It acts as a reverse proxy, redirecting and distributing the requests from the clients to the servers.

Load balancers can be placed not only between clients and servers, but in many other different places of a system like between servers and databases or at DNS layer when dealing with web servers.

There are different types of load balancers and multiple way to categorize them.

Hardware and Software Load Balancers

A first categorization distinguishes between hardware and software load balancers.

Hardware load balancers are high-performance devices able to process high traffic from many kind of applications. They can also have the capabilities to support virtual load balancers instances on the same hardware.

They are efficient, but also expensive. Popular vendors are Citrix or Radware.

Layer 4 and Layer 7 Load Balancers

A second categorization distinguishes between Layer 4 and Layer 7 load balancers, according to the network layer where they act.

Layer 4 load balancers redirect traffic based on the TCP or UDP ports of the packets along with their source and destination IP addresses. Instead, layer 7 load balancers redirect traffic inspecting the actual content of each packet, including HTTP headers and SSL session IDs.

Servers selection strategies

There are still two important questions to answer. The first one is how a load balancer knows to which servers distribute the traffic. The second one is how a load balancer distributes the requests to the servers.

The first question is easier to answer. The system administrators configure load balancers and servers to know about each others. When they add or remove a new server, this is registered or deregistered with the load balancer.

The second question is more complex. There are many different server selection strategies. Let's revise the most popular ones.

Random redirection: the load balancer redirects the traffic to the servers following a pure random order. This strategy could have problem because a server can get overloaded by chance.
Round Robin: the load balancer evenly redirects the traffic to the servers following a certain circular order. A variant is the weighted Round Robin where each server receives a number of requests according to a given weight. This can be useful if some servers are more powerful than others.
Performances based redirection: this is a more involved approach. The load balancer performs healthy checks on the servers to get statistics like how much traffic they are handling any given time or how long they take to answer or how many resourcers they are using. The load balancer redirects the traffic according to this information. If the load balancer gets to know that a server is doing poorly, it stops redirecting request to that server. Instead if the load balancer notices that a server is not overloaded and performs well, it redirects more traffic to it.
IP based redirection: the load balancer calculates a hash of the client's IP address and redirects the traffic according to the value of the hash. This can be useful if the server caches the results of the requests. Why? Becaues all the requests of the same client are always redirected to the same server in which the responses of the particular client has been cached. So cache hits get maximized.
Path based redirection: the load balancer distributes requests according to their URL. Why can this be useful? All the requests for a service are redirected to a specific set of servers. This separates the services making easier the deployment and isolating them in case of failures.

You could now wonder which strategy is better. The answer is no one. Each strategy has advantages and disadvantages and it's always wise to select the best strategy fitting your use case. In absence of clear requirements, Round Robin is always a good strategy to start with.

It can also make sense to have multiple layers of load balancer adopting different strategies. The load balancer in the first layer redirects the traffic to the load balancer in the second one that then redirects the traffic to the servers. For example the first layer could use the hash and second the Round Robin.

Redundancy

Load Balancers can be single points of failure in a system. If you have a single load balancer and this gets overloaded or fails, the whole system goes down.

In order to prevent this, multiple replicas of the load balancer are usually connected into a cluster. Each replica monitors the healthy of the others. In case the main load balancer fails, another replica takes over and starts rerouting the traffic.

Advantages

These are the most straightforward advantages you can get introducing the use of load balancers while designing your system:

Optimizing the use of resources by distributing clients requests across servers
Improving the user experience by ensuring a faster response time
Preventing system downtime by rerouting requests from overloaded/failed servers to healthy ones
Giving more flexibility in adding/removing servers and performing server maintenance
Introducing more redundancy and resilience into the system

Clustering

Load balancing shares common traits with clustering but there is a main difference. Servers in a cluster are aware of each other and work toward common purpose. Load balanced servers just react to commands from LB and don't know about each other.

Conclusion

Load balancers are a key component of any distributed system and can be extremely useful while designing a system. They can be placed in many place of a system, increasing its flexibility, improving its performances and making it more robust. I hope this post helped you in understanding better what load balancers are, what are the different kind of load balancers and how you can use them while designing a system.

If you liked this post, follow me on Twitter to get more related content daily!

All what developers need to know about proxy servers

2021-10-09T00:00:00+00:00

Proxies are a fundamental component of almost every distributed system. The average person has only a vague understanding of the purpose of proxy servers and often associate them with unblocking some media content from other countries. As a developer however, you should know that proxy servers do much more and are vital for businesses.

This post will explain you what proxies are, which are the different kind of proxies and why they are so useful when designing a distributed system.

What is a proxy?

Let's start with a general definition. In real life a proxy is a person who is given the power or authority to do something for someone else. For example, if you are not available to vote, you can nominate another person to act as your proxy and vote for you.

In distributed systems, a proxy is a server acting as intermediary between a set of clients and a set of servers. We can distinguish 2 categories of proxies:

forward proxy: the proxy acts on behalf of the clients
reverse proxy: the proxy acts on behalf of the servers

Forward Proxies

A forward proxy is a server that acts on behalf of the client. If the proxy has been configured correctly, when a client want to communicate with a server the requests from the client don't go directly to the server but to the proxy.

The proxy then performs the following steps:

forward the requests to the proper server
wait for responses
send the responses back to the corresponding client.

Notice how in this way the server doesn't know who is the client. It only knows the source IP address of the proxy and not of the client. So a forward proxy can secure and hide the identity of the client. This is basically the same principle how Virtual Private Networks (VPN) work.

What are the main advantages of using a forward proxy?

the clients could use a proxy to access servers that they would not be supposed to access otherwise
a system administrator could configure a proxy to selectively send/block certain requests from the client (i.e. allow or block the access to web pages of specific sites)
a proxy can log or monitor requests
a proxy can cache responses (i.e. frequently used pages so the user request doesn’t have to retrieve them each time from the web)

Reverse Proxies

A reverse proxy is a server that acts on behalf of another server. If the reverse proxy has been configured correctly, the requests of the client are not received directly by the server, but by the reverse proxy.

Once again, the next steps are quite obvious. The reverse proxy forward the requests to the proper server, wait for responses and send the responses back to the corresponding client.

Notice how the client has no idea that it is communicating with the reverse proxy. The responses are then returned to the client, appearing as if they were originated from the server itself. So a reverse proxy can secure and hide the identity of the server.

Let's consider a practical example to understand better. Let's suppose that you type http://francofernando.com in your browser. Your browser makes a DNS query to get the IP address of my website. If my website used a reverse proxy, the DNS would return the reverse proxy IP address instead that the one of my website.

What are the main advantages of using reverse proxies?

Security and anonymity: by intercepting requests directed to the backend servers, a reverse proxy can not only protect their identities but it can also act as an additional defense against security attacks. A reverse proxy can also make possible the access to multiple servers from a single URL regardless of the structure of the local area network.
Performance: Reverse proxies can cache static content reducing the overall system latency. For example they can cache HTML/CSS/JS, photos and videos. They can also take the load off of the servers performing additional tasks like: data compression and decompression, data encryption and decryption, HTTP authentication.
Load balancing: Reverse proxies can distribute client requests across a group of servers in a way that maximizes speed and capacity utilization, ensuring that no one of the servers gets overloaded.

Introducing reverse proxies in a system have also some disadvantages. The main one is that the complexity of the overall system increases. Not only a reverse proxy represents an additional component introduced into the sysytem, but it is also represents a single point of failure. So it is usually necessary to make it redundant and configuring multiple reverse proxies further increases the complexity.

As always when designing a system, deciding if use or not a reverse proxy layer is a matter of trade offs.

Some examples of reverse proxy implementation are Nginx, Caddy, HAProxy, Squid.

Forward vs Reverse Proxies

Notice how forward and reverse proxies have an exact opposite interaction pattern.

Forward proxies are configured by the clients. They send out requests on behalf of the clients and receive responses from the servers.

Reverse proxies are configured by the system administrators. They receive requests on behalf of the server and send out responses to the client.

API Gateways

In microservices architectures, reverse proxies can also act as API Gateways. An API gateway is a single entry point for external clients that implements an abstraction layer hiding all the details about microservices.

The API gateway handles the requests from the clients in one of two ways:

routing the requests directly to the appropriate services
dispatching the requests to multiple services and aggregating the results

Rather than provide a one-size-fits-all style API, the API gateway can also expose different APIs for different clients. It's not uncommon also to have different API gateways for web application, mobile application, and external 3rd party applications.

API gateways provide many benefits:

reduce the volume of requests and traffic, combining multiple API calls
allow a flexible implementation of internal microservices, decoupling them from the clients
make easier collect log, metrics and monitoring the activities

Conclusion

Proxy servers are a fundamental component of distributed systems and can be extremely useful in a lot of scenarios. I hope the post helped you in understanding better what proxy server are and which are the different kind of proxies you can use while designing a system.

If you liked this post, follow me on Twitter to get more related content daily!

Three effective ways to create modulo n integer sequences.

2021-08-27T00:00:00+00:00

A modulo n integer sequence is just a (potentially infinite) integer sequence that wrap to 0 when the current value is n. For example, $0,1,2,3,4,0,1,2,3,4,...$ is a modulo 4 integer sequence.

Modulo-n integer sequences are widely used in programming. For instance, they are necessary in any array-backed data structure (e.g. ring buffers or fixed size queues) in order to reuse elements when the end of the array is reached.

In this post, we will shortly review three different ways to create such sequences examining their pros and cons.

Integer modulo operator

Let's assume that the current value of the sequence is stored in an integer variable called counter. The traditional way to generate the next value is to increase counter, divide the result by n and then keep the remainder using the modulo operator.

cnt = (cnt + 1) % n;

This solution is easy and readable, but not so performant. Indeed, the integer modulo operator is expensive and requires far more clock cycles than other operations like addition or subtraction.

Comparison

A second way is to replace the modulo operator with a comparison. We reset counter to 0 when it is equal to n and we increment counter otherwise.

  cnt = (cnt + 1);
  if (cnt >= n) cnt = 0;   

This approach is usually more efficient than the previous one even if the performance benefits is completely platform dependent.

Modulo with bitwise and operator

If n is a power of two (i.e. n = 2^m), it is possible to create a modulo-n sequence in a very efficient way using the bitwise and operator.

Under this special condition indeed

cnt = (cnt + 1) % n;

becomes equivalent to

cnt = (cnt + 1) & n;

But why this approach works?

When n = 2^m, we have that:

the modulo operator takes only the least significant m bits of the counter variable
n-1 is a number having the last m bits equal to one (i.e. n = 64 = 1000000 in binary, n-1 = 63 = 111111)

So if you consider the bitwise and table of truth, it becomes clear how the bitwise and operator can be used to implement the modulo operator in this particular case.

This third method is very efficient because the bitwise and operator is really fast. The downside is that the method is not applicable to all use cases and it requires some binary math knowledge to be understood.

The following example shows how to generate a modulo-64 sequence using the AND operator:

  cnt = (cnt + 1) & 0x3F;  // n = 64, m = 6

Conclusion

We reviewed three different methods of generating modulo n sequences of integer. I would generally suggest to use one of the first two methods, since they are more readable and appicable without any restriction. Anyway, if you need a very performant code, you could consider also the third option. For example, if you are implementing a circular buffer, you might consider setting its size to a power of two speed up performance on accessing the buffer. Since I used C++ as reference language, in the code snippets the modulo operator has been represented by % and the biwise and operator by &. If you liked this post, follow me on Twitter to get more related content daily!

The complete guide to the Disjoint Set Union data structure.

2021-06-29T00:00:00+00:00

The Disjoint Set Union (DSU) is a data structure representing a collection of disjoint sets. It is extremely helpful in solving problems where it is required to track the connectivity of elements belonging to a specific subset and to connect different subsets with each other. Each subset is represented in the form of tree and the root of the tree is the representative element of the set.

The data structure provides the following operations:

Find(x): find the representative of the set in which the elements x is located
Unite(x, y): merges the subsets in which the elements x and y are located
Same(x, y): determines if the elements x and y are located in the same subset

Data Representation

Instead of representing the subsets using classical trees with nodes and link to child nodes, it is convenient to use an array. Assuming that each of the n elements is associated to an index between 0 and n-1 (e.g. using an hash table), we can use an array of size n to store the index of the parent of each element in the subset. A special case is the representative of each subset which contain its own index. The elements of the array are initialized with their own index, since every element is the representative of its subset before any Unite operation has been performed.

class dsu{
        
        vector<int> parent;

        public:
        
        dsu(int n) {
            parent.resize(n);
            iota(parent.begin(), parent.end(), 0);
        }

        //public methods

Naive Implementations: quick find and quick union (O(N))

The easiest implementation of DSU assumes that the representative of each element is its parent. This leads to define DSU operations as follows:

Find(x): return the parent of x
Unite(x, y): assign x as parent to all elements having y as parent
Same(x, y): check if x and y have the same parent

This implementation is also known as quick find since the Find operation is performed in constant time. The Unite operation takes instead linear time since every time the entire parents array needs to be traversed.

int find(int x) {            
  return parents[x];
}

void unite(int x, int y) {

  int rootX = find(x);
  int rootY = find(y);
  
  for (int& parent : parents) {
    if (parent == rootY) {
      parent = rootX;
    }
  }
}

An alternative implementation assumes that two elements belong to the same set only if they have the same root. This leads to define DSU operations as follows:

Find(x): return the root of x
Unite(x, y): assign root[x] as parent to root[y]
Same(x, y): check if x and y have the same root

Also in this case the Find operation takes in the worst case linear time because it requires to visit all the ancestors of the given element until the root has not been reached. Anyway, on average the Find performs better because not always the entire parents array needs to be traversed. This implementation is also known as quick union since the Unite operation takes on average less than the one in the quick find.

int find(int x) {
            
  while(parents[x] != x) {
      x = parents[x];
  }
  return x;
}

void unite(int x, int y) {

  int rootX = find(x);
  int rootY = find(y);
  
  if (rootX != rootY) {
    parents[rootY] = rootX;
  }
}

Weighted Union (O(logN))

In the quick union, the Unite operation always attaches the root of the tree of the second element to the root of the tree of the first one. The Weighted Union implementation of the DSU modifies the Unite operation so that the root of the tree having less elements is always attached to the root of the tree having more elements. This simple optimization allows to keep each tree balanced, reducing the time complexity of both the Unite and Find operations to logarithmic. The price to pay for this is the additional space required to keep track of the size of each tree.

void unite(int x, int y) {

  int rootX = find(x);
  int rootY = find(y);
  
  if (rootX == rootY) return;
  
  if (size[rootX] < size[rootY]) swap(rootX, rootY);
  size[rootX] += size[rootY];
  parents[rootY] = rootX;
}

Path Compression (O( $\alpha N$ ))

Path compression is an optimization designed to speed up the Find operation making the path between an element and its root shorter after every execution. There are two main ways to implement such optimization. The first one attaches each element to his grandfather, halving the length of the path between the input element and the root. The second one attaches all the elements on the path directly to the root. The first variant can be easily implemented iteratively, the second one recursively.

//iterative
int find(int x) {
    
    while (parents[x] != x) {
        x = parents[x];
        parents[x] = parents[parents[x]];
    }
    return parents[x];
}

//recursive
int find(int x) {
            
  if (parents[x] == x) return x;
  return parents[x] = find(parents[x]);
}

Combining path compression with weighted union, we can reduce the time complexity of both the Unite and Find operations nearly to constant. Indeed, it is possible to demonstrate that the final amortized time complexity (i.e. the total time per operation, evaluated over a sequence of multiple operations) is O( $\alpha(N)$ ), where $\alpha(N)$ is the inverse Ackermann function. Such function grows so slowly, that it doesn't exceed 4 for all reasonable values of N.

Applications

The Disjoint Set Union can significantly reduce the execution time of an algorithm when used in the right context and has many application.

The simpler and most obvious one is to determine the connected components in a graph. For example let's consider the following problem: we are given an unsorted array of N integers (i.e. [9,15,2,6,14,8,4,7]) and we want to compute the length of the longest consecutive elements sequence (i.e. [6,7,8,9] with length 4). The brute force algorithm takes O(N³) time: for each element try to build the longest sequence first on the right and then on the left. A better solution takes O(Nlog_N) to sort the array and then find the longest sequence with a linear scan.

The DSU solution outperforms both: once we notice that each consecutive sequence can be considered like a connected component, the solution is straightforward. We just need to use an hash table to map number in the sequence to an integer between 0 and N-1 and add an api to the disjoint set data structure to return the size of the largest connected component. You can check the whole code here for more details.

int longestConsecutive(vector<int>& nums) {
        
  unordered_map<int,int> numToPos(nums.size());
  for (int i = 0; i < nums.size(); ++i) numToPos[nums[i]] = i;
  
   //initialize parents and weights
  dsu dsu(nums.size());
  
  for (int num : nums) {
      if (numToPos.count(num-1) > 0) dsu.unite(numToPos[num], numToPos[num-1]);
      if (numToPos.count(num+1) > 0) dsu.unite(numToPos[num], numToPos[num+1]);
  }
  
  return dsu.maxSize();

Another classical application can be derived from image processing. We are given an image of NxM pixels and we want to determine the size of each connected component having pixels with the same intensity value.

Using DSU, we can do these simply steps:

iterate over all the pixels in the image;
for each pixel iterate over its four neighbors;
perform a Unite operation if the neighbor has the same intensity value.

This approach has also the advantage to process the matrix row by row (i.e. to process a row we only need the previous and the current one) using O(min(N,M)) memory.

However, the most famous application of the DSU is its usage as subroutine of the Kruskal algorithm which finds the minimum spanning tree of a weighted graph. A spanning tree is a subgraph which connects all the vertices and the minimum spanning tree has minimum sum of weights of all its edges. The idea is simple: initially all the vertices are considered as trees isolated from each other and the edges are sorted by weight in non-decreasing order. Then the trees are gradually merged considering an edge at each iteration and combining the two connected trees.

Here the DSU data structure can help to check if the two vertices of each edge are not belonging already to the same tree in order not to introduce a cycle in the spanning tree. The use of DSU enables the algorithm to achieve a time complexity of O(NlogM) instead of O(NlogM + N²), where N is the number of vertices and M the number of edges.

struct Edge {
    int u, v, weight;
};

vector<Edge> Kruskal (int n, vector<Edge> const& edges) {

  //int cost = 0;
  vector<Edge> result;

  //initialize parents and weights
  dsu dsu(n);
 
  //lambda...
  sort(edges.begin(), edges.end(), [](auto const& lhs, auto const& rhs)
  {
    return lhs.weight < rhs.weight
  });

  for (Edge e : edges) {
    if (dsu.find(e.u) != dsu.find(e.v)) {
        //cost += e.weight;
        result.push_back(e);
        dsu.unite(e.u, e.v);
    }
  }
}

Conclusion

The Disjoint Set Union is a fast and simple data structure that every developer should know about since it comes in handy in many application. I hope this article helped you in solidify your knowledge of this data structure and in understanding how the implementation details have here a big impact on the asymptotic performances. You can find in my Github repository a complete implementation of the DSU in multiple programming languages.

If you liked this post, follow me on Twitter to get more related content daily!

Docker Laboratory

2021-06-14T00:00:00+00:00

This post goes through a set of exercises and tasks regarding the Docker platform. The goal is getting familiar with Docker and with the most frequently commands of the Docker CLI. This post requires a basic knowledge of Docker. You can refer to this other post for an introduction to Docker. All the material used in this post is freely available at my Github repository.

Running a container

Let's start launching a fully functional WordPress blog engine using a Linux-based Docker container, so that we can learn the commands needed to pull a container image and then launch the container using the Docker CLI.

Type the command docker pull tutum/wordpress to connect to the public Docker Registry and download the latest version of the WordPress container image published by tutum (hence the format tutum/wordpress). The image will be pulled from the registry layer by layer.
Type the command docker run -d -p 80:80 tutum/wordpress The option -p is used to map the port 80 of the container (second port) to the port 80 of the docker host (first 80). So if we want to run multiple Wordpress images as separated containers we need to map the other containers port to different host ports (i.e. docker run -d -p 8080:80 tutum/wordpress ). The -d option tells Docker to detach and run the container in the background. We can also run a container with a name, using the –name option (i.e. docker run –name mywordpress -d -p 8081:80 tutum/wordpress). If we don't do this a random name is assigned.
Use the command docker ps to visualize running containers.
Open a browser and navigate to the URLs localhost:80 and localhost:8080 to verify that the WordPress blog instances are actually running.

Working with the Docker Command Line Interface (CLI)

Let's now use some common Docker commands needed to work with containers.

We can stop a running container by using the docker stop CONTAINER_ID command. Here CONTAINER_ID is the identifier of a running container. We can usually just use the first couple characters to identify the container ID. We can then use again the docker ps command and notice how the listing shows one less container running. The -a option allows to show all containers, even those that are stopped.
We can also issue a command to start the container which was stopped: docker start CONTAINER_ID. We can then check that container has started successfully running again the docker ps command.
Stopping a container does not remove it. To delete/remove a container and free the resources we need a different command: docker rm -f CONTAINER_ID. The -f option is used to force the remove operation and it is needed if we are trying to remove a container that is not already stopped.
It is also possible to stop all the running containers together using docker stop $(docker ps -aq). Here we are issuing two commands: first we use the docker ps with relevant options to capture the list of container IDs and then we pass this list of IDs to the docker stop command.
Removing a container does not remove the underlying image but only the specific container that was based on the image. To remove the image and reclaim its resources, like disk space, we need to issue a different command: docker rmi IMAGE_ID -f. The -f is to force the removal. We cannot remove an image associated with a stopped (but not removed) container unless we use the -f parameter. To retrieve the image ids we can use the command docker images.

Building and Running Custom Container Images with Dockerfile

A Dockerfile is essentially a plain text file with Docker commands in it. We can think of it as a configuration file with a set of instructions needed to assemble a new Docker image. Here we will try to build and run ASP.NET Core 3.x Application Inside A Container. All the necessary files are available in my github repo.

Inspect the Dockerfile to understand which instructions will be executed. Multistage Dockerfile…

FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build-env
WORKDIR /app

# Copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore

# Copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out

# Build runtime image
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "mywebapp.dll"]

Create the container image running the command docker build -t myaspnetcoreapp:1.0 .. Here the -t option assign the 1.0 tag the new image to indicate its version. The final dot is also important and means to use the Dockerfile in the local directory.
Launch the container to run our application using the command docker run -d -p 8090:80 myaspnetcoreapp:1.0. We are now running our ASP.NET Core application inside the container listening at port 80 which is mapped to the port 8090 on the host.
Open a browser and test the application using the address localhost:8090.

Interact with a running container

In some situations we need to interact with a running container for the purposes of troubleshooting, monitoring etc.

We can establish an interactive session a with a running container using its CONTAINER ID or NAME with the command docker exec -it CONTAINER_ID_OR_NAME bash
A new interactive session is now establish to a running container. Since "bash" is the program that was asked to be executed you now have access to full bash shell inside the container. We can run a command "ls" to view the listing of files and directories. Notice it has all the files copied by Dockerfile command in previous section.

Making Changes to a Running Container

While we are interacting and running commands inside a running container, we may also want to make changes/updates to it and create a brand-new image out of these changes.

Establish an interactive session with our aspnetcore container as showed in the previous section.
Run the command and then apt-get update and then apt-get install nano to install the nano text editor inside the running container.
Run the command nano appsetting.json to open the appsetting.json file for editing.
Change the Config1 text message to something else. Then exit the nano editor pressing "CTRL + X" and then pressing "Y" when asked for confirmation to retain changes. Then press enter without changing the name of the file. This will close the Nano text editor.
Exit the interactive bash session using the the command exit
The running container needs to be stopped first and then started again to reflect the changes. So stop the running container and start it again as we already did in the previous sections.
Verify that the changes to the container were persisted opening a browser and typing the address localhost:8080.

Persist Changes to a new Image

The changes we made in the previous section are only available to that container and if we were to remove the container, these changes would be lost. One way to address this is by creating a new container image based on running container that has the changes. This way the changes will be available as part of a new container image. Even if this is helpful during development/test phases, where rapid development and testing requires a quick turn-around time. this approach is generally not recommended because it is hard to manage and scale at the production level.

Create a new image running the command docker commit CONTAINER_ID myaspnetcoreapp:2.0. Here the docker commit command is used to create a new image from a container's changes.
View the list of all container images to check that the new image is available and then run the new image to test it. After opening a browser we should see agsin the updated message displayed.

Working with multiple containers

Working with multiple containers interacting together is very common scenario. So let's try this out using a web application that runs in a container and talks to a database hosted in another container. We will use an Azure SQL Server instance as database and an ASP .NET Core Application as web application.

Inspect the Dockerfile describing how to package the database. The entrypoint is the script entrypoint.sh which will start the database server and run import-data.sh. The import-data.sh script will then wait for the server to start and will trigger the database creation, the data import and the ping command to keep the database alive. The SQL commands necessary to create a database are defined in the setup.sql file, while the Users.csv file contains the test data.
```
FROM mcr.microsoft.com/azure-sql-edge:latest

# Create db directory
RUN mkdir -p /usr/src/db
WORKDIR /usr/src/db

# Install ping that will be used to keep the container up
RUN apt-get update && apt-get install -y iputils-ping

COPY . .

# Grant permissions for the import-data and entrypoint scripts to be executable
RUN chmod +x /usr/src/db/import-data.sh
RUN chmod +x /usr/src/db/entrypoint.sh

CMD ./entrypoint.sh
```
Run the command to build our Azure SQL Server container image docker build -t myazuresqlserver ..
Run the container with the command docker run -e ACCEPT_EULA=Y -e SA_PASSWORD=P@ssw0rd123! -d -p 1433:1433 –name mydb myazuresqlserver. This command takes some environment variables passed with the -e option. The ACCEPT_EULA variable confirms our acceptance of the End-User Licensing Agreement, the SA_PASSWORD variable is the password used to connect to Azure SQL server once the container is running. We can follow the database initialization with the command docker logs mydb -f until we see the ping command starting. Once it started, we can interrupt the logs by pressing CTRL + C.
Run the command docker inspect mydb and note down the IP Address of the server. We need to know what is the IP address of the container running the server so that the web application can connect to it.
Change the working directory to the one of the web application and modify the Startup.cs file replacing localhost with the ip address of the server.
Build the ASP .NET Core web application using the command docker build -t myaspnetcoreapp:1.0-withsql .. Then run the container and expose the port 8082 with the command docker run -d -p 8082:80 myaspcoreapp:3.1-withsql.
Open a browser and naviguate to the address localhost:8082. The web app should display the list of users defined in the database.
Click on Create New Users and add a new user name.
Run a new instance of the container and expose a different port (i.e. 8083) with the command docker run -d -p 8083:80 myaspcoreapp:3.1-withsql. Opening a browser at the address localhost:8083 we still see the new user that we have created since it is stored in the SQL Server container.

Working with Data Volumes

Docker containers are volatile by design and don't guarantee data persistence. This means that if we remove a container all the data that was in the container (running or stopped) will be lost. This certainly causes a challenge for applications that are running in the container and need to manage a state. A good example here would be the SQL Server Database we previously used. This file it is required to be persisted beyond the life of the container running the SQL engine. The standard solution is to use data volume to persist data independently of the container's lifecycle. Volumes are initialized when a container is created and they are never automatically deleted, even if they are no longer referenced by a container. This means we are responsible for cleaning up volumes ourself.

As a first step, let's see how we can mount a host directory as a data volume.

Create a directory on the host operating system called "HostData".
Run an Ubuntu container in interactive mode and mount the host directory as a data volume with the command **docker run -it -v :/home/Data/ ubuntu bash**. Here the *-v* option mount the host directory HostData inside the container as /home/Data/. We can also choose the same name for the directory inside the container and the host but it's not mandatory.
Check that the /home/Data/ directory exists inside the container by running the command ls /home.
Create a file in the /home/Data/ folder and add more text to it. Running the command: **echo File is updated by container: >> /home/Data/message.txt**. The can be retrieved running the command **hostname**.
Look at content inside message.txt by running the command cat /home/Data/message.txt
Exit the interactive mode using the exit command and run the command **docker inspect **. Check the the section labeled "Mounts" to see the details about the mounted volume.
Run another container in interactive mode and mount the host directory as a data volume running again the command ç.
Add more text to the /home/Data/message.txt file with the command **echo File is updated by container: >> /home/Data/message.txt**. Then look at content inside message.txt by running the command **cat /home/Data/message.txt**. The file should contain two update messages: this one and the one of the stopped container.
Open the HostData/message.txt in the Host with a text editor and update again the file. Then go back to the Docker interactive session and use again the command cat /home/Data/message.txt. The file should also contain the update message from the host. Then exit from the interactive mode.

Mounting a directory on the host as a data volume is a very handy way to share the content from host to container, but it's not ideal in terms of portability. If we later run the container on a different host,there is no guarantee that this host will have the same directory. This would cause the application inside the container to break as it depends on a content that does not exist in the host. When a higher level of portability is desired, we can mount a shared storage volume using a volume plugins. The benefit of using shared volumes is that they are host-independent: a volume can be made available on any host on which a container is running as long as the container can access to the shared storage backend and has the plugin installed. Let's try to create and use a shared-storage volume using the local driver which uses local host for the storage. The exact same procedure will work with production ready storage drivers.

Create a volume by running the command following command docker volume create -d local myvolume
List all the volumes by running the command docker volume ls. The myvolume volume should be available as a local driver.

Run the command docker inspect myvolume. Here Mountpoint is set at a location on the C drive under the ProgramData\docker folder. This is the default location for local storage drivers. Using another commercial storage driver, the location will be different.
Launch a container and make that storage volume available inside the container running the command docker run -it -v myvolume:/home/Data/ ubuntu bash.
Check that the /home/Data/ directory exists inside the container by running the command ls /home.
Create a file in the /home/Data/ folder and add more text to it. Running the command: **echo File is updated by container: >> /home/Data/message.txt**.
Look at content inside message.txt by running the command cat /home/Data/message.txt, then exit the interactive mode.

Docker Networking

Let's see some Docker CLI commands to view Docker default networks and create a custom bridge network.

Run the command docker network ls to view the list of networks available to docker. The 'bridge' network is the default network for containers running on Linux. Any containers that run without any flags or arguments to implement specific network configurations will be attached to the default 'bridge' network and automatically assigned an IP address from the 'bridge' network's internal prefix IP range.
View detailed information about the Docker default bridge network using the command docker inspect bridge. The "Containers" key refers to all containers that are using the specified networkand it is empty because there are no containers currently running.
Launch a new container by running the command docker run -d adiazmor/docker-ubuntu-with-ping ping localhost. Run again the command docker inspect bridge and check that the "Containers" key now includes information regarding the container that is using the bridge network.
Create a new docker network by running the command docker network create -d bridge –subnet=192.168.15.0/24 –gateway=192.168.15.1 custom-bridge. Here the -d flag stands for network driver and specifies the network type we want to create ("bridge"in this case). We are also providing the IP prefix and gateway address using the –subnet and –gateway option.
Use the docker network ls command and check that the "custom-bridge" network is available.
Launch a new container using the new custom bridge network with the command docker run -d –network=custom-bridge adiazmor/docker-ubuntu-with-ping ping localhost. Here the –network option force docker to use specific network for the container. Then use the docker network inspect custom-bridge command to get the detailed information about custom-bridge network and the container that is using it. Notice how the subnet and gateway values reflect the values we used during the creation of the network. Also note that the container IPv4 Address, 192.168.15.2 is in the custom-bridge network.
Remove all the containers with the command docker rm (docker ps -aq) -f. Then remove the custom-bridge network with the command docker network rm custom-bridge. Check that only bridge network remains with the command docker network ls.

Working with Docker-Compose

Docker-compose is a tool for defining and running multi-container Docker applications. To use compose, we first need to create a .yml file describing which containers we want to start, the corresponding images name and the registry name. Once this file has been created, we can then use a single command to start and stop all the containers. Let's see how Docker-compose work using a simple "Famous Quotes" application that consists of a frontend web app that talks to a RESTful API to fetch quotes in JSON format. Both the web app and API are developed using ASP.NET Core and each will run in a separate container.

Inspect the docker-compose.yml file. This file will ensure that the web API can be accessed by the web application without hardcoding its domain name or IP Address. This is not necessary since docker-compose will make these services discoverable. The file also defines specific dependencies, such as the web app container will depend on the web API container.

# Docker-compose version
version : '3'

services:
demowebapp:
    # Build instruction pointing to the folder containing the web application
    # and its dockerfile. This is equal to the docker build command for that application. 
    build: ./mywebapp 
    # Map host port 80 to container port 80
    ports:
    - 80:80
    ## Express dependency between applications
    depends_on:
    - demowebapi
demowebapi:
    build: ./mywebapi
    # Map host port 9090 to container port 9090
    ports:
    - 9000:9000

Run the multi-container application with the single command docker-compose up -d. Here the -d option works the same as when used with the docker build command instructing docker to run the container in the background rather than interactively.
Check details about running docker-compose services by executing the command docker-compose ps
Open a web browser and browse to localhost to land on the home page of web application. Then test the Web API selecting the Quotes option from the top menu bar. This will generate a call to web API and the results will be displayed on the web application.
Stop and remove the multi-container application with the single command docker-compose down -d. If we only wish to stop the multi-container applications and the associated running containers without removing them we cab use the docker-compose stop command instead.

Docker allows us to create custom networks. Let's try to create and configure a custom bridge network replacing the default one.

Create a new docker network by running the command docker network create -d bridge –subnet=192.168.15.0/24 –gateway=192.168.15.1 custom-bridge.
Use the docker network ls command and check that the "custom-bridge" network is available.
Launch a new container that uses the new custom nat network with the command docker run -d –network=custom-bridge adiazmor/docker-ubuntu-with-ping ping localhost. Here the –network option force docker to use a specific network for the container. Then use docker network inspect custom-nat command to get the detailed information about custom-nat network and container that is using it.

Conclusion

If you have successfully completed all the tasks and exercises proposed in this article, congratulation! You are now certainly more familar with the Docker CLI and ready to experiment yourself.

How to efficiently compare strings with rolling hash

2021-05-16T00:00:00+00:00

Hashing algorithms are useful to solve a lot of problems. In this post we will discuss the rolling hash, an interesting algorithm used in the context of string comparison.

The idea is to have a hash function computed inside a window that moves through the input string. When the window moves, this hash function can be updated very quickly without rehashing the whole window using only:

the old hash value;
the old character removed from the window;
the new character added to the window.

Let's see how this works under the hood using a specific class of hash functions known as polynomial hash.

Polynomial Hash Function

Given an input string s of length n, we can compute its polynomial hash with multiplications and additions according to the following formula:

H(s) = s\left[0\right]\cdot b^{n-1} + s\left[1\right]\cdot b^{n-2} + ... + s\left[n-1\right]\cdot b^{0} = \sum_{i=0}^{n-1}s[i]\cdot b^{n-1-i}

where b is a constant positive number. Since all the characters in the input alphabet can be reduced to a positive number between 0 and the size of the alphabet, this formula is equivalent to interpret the string s as a number in a numeral system with base b.

It is reasonable to chose b as the first prime number greater or equal to the number of characters in the input alphabet. The choice of a prime number will guarantee a more uniform distribution of the keys and a lower number of collisions, regardless the nature of the input strings.

For example if the input is composed of only lowercase letters of the English alphabet, we can take b=31. With this choice the string bus has the following polynomial hash:

H(bus) = (b-a) \cdot 31^{2} + (u-a) \cdot 31^{1} + (s-a) \cdot 31^{0} = 1 \cdot 961 + 20 \cdot 31 + 18 = 1599

A naive implementation computing the polynomial hash of a string s is the following (the reverse iteration is for consistency with the formula):

int hash(string const& s) {
	const int b = 31, m = 1000000009;
	int h = 0;
	int b_pow = 1;
	for (auto it = s.crbegin() ; it != s.crend(); ++it) {
		//Converting a to 0 the hashes of the strings a, aa, aaa
		//all evaluate to 0, so add 1
		h = (h * b_pow) + (*it - 'a' + 1);
		b_pow *= b;
	}
	return h;
}

Modular Arithmetic and Horner's Rule

Calculating a polynomial hash function naively has two major drawbacks:

accuracy, since manipulating large powers of b can cause integer overflow;
inefficiency, since each character requires two multiplications and one addition.

The risk of integer overflow can be removed using modular arithmetic and performing all operations modulo M. According to the principles of modular arithmetic

$(a + b)\mod M = (a \mod M + b\mod M)\mod M$
$(a \cdot b)\mod M = (a \mod M \cdot b\mod M)\mod M$

we can rewrite the implementation of the polynomial hash as follow:

int hash(string const& s) {
	const int b = 31, M = 1000000009;
	int h = 0;
	int b_pow = 1;
	for (auto it = s.crbegin() ; it != s.crend(); ++it) {
		//Converting a to 0 the hashes of the strings a, aa, aaa
		//all evaluate to 0, so add 1
		h = ((*it - 'a' + 1) + h * b_pow) % M;
		b_pow = (b_pow * b) % M;
	}
	return h;
}

Clearly, the value of M shall be large enough to reduce the number of hash collisions. To have a better distribution of the keys, it is usually reasonable to chose M as a large prime number.

The efficiency of the computation can be improved using the Horner’s rule, a popular method that simplifies the evaluation of a polynomial of degree n by dividing it into n monomials (polynomials of degree 1). With the Horner’s rule the polynomial hash formula can be written as

H(s) = s\left[n-1\right] + b \cdot (s\left[n-2\right] + b \cdot (s\left[n-3\right] + ... + b \cdot (s\left[1\right]\cdot + b \cdot s\left[0\right])))

and evaluated with only n multiplications and n additions. This is optimal, since a polynomial of degree n that cannot be evaluated with fewer arithmetic operations.

Using the Horner's rule with modular arithmetic, the polynomial hash of a string s can be computed as follows:

	
int hash (string const& s) { 
		
	const int b = 31, M = 1000000009;
	int h = 0;
		
	for (char c : s) {
		h = (h*b + (c-'a'+1)) % M; 
	}
		
	return h;
}

Rolling Hash

Let's now consider a sliding window of length 3 inside the string "business" and let's suppose that we already computed the hash value inside the starting window "bus". If the window move one position to the right, how can we generate the hash for the substring "usi" in constant time? The only changes from one window to the next one are the removing of the first letter 'b' and the adding of the last letter 'i'.

In terms of polynomial hash function this means:

H(usi) = (H(bus) - (b-a) \cdot 31^{2}) \cdot 31 + (i-a) \cdot 31^{0} = (1599 - 961) \cdot 31 + 8 = 19786

Generalizing, we can get the new hash by

subtracting the removed character $C_{prev}$ to the old hash value $H_{old}$ : $H_{old} - (C_{prev}-a) \cdot b^{n-1}$
multiplying the updated hash value for b, so that each remaining character has the right power of b
adding the next character $C_{next}$

H_{new} = (H_{old} - (C_{prev}-'a') \cdot b^{n-1}) \cdot b + C_{next}

	
int roll (string const& s, int previousHash, char prevChar, char nextChar, int bPow, int M) { 
		
	previousHash -= (bPow * prevChar - 'a') % M;
	previousHash *= bPow;
	previousHash += (nextChar - 'a');
	return previousHash;
}

Rabin Karp Algorithm

The most known application of the rolling hash is the Karp-Rabin algorithm which solves the following problem: given two strings, a text t and a pattern p, determine whether p appears in t. The brute force solution is straightforward: for every position in t, consider it as a starting position of p and check if we get a match.

bool bruteForce(string const& text, string const& pattern) {
	
	for (int i = 0; i < text.size(); i++) {
		for (int j = 0; j < pattern.size() && (i + j) < text.size(); j++) {
			// mismatch found, break the inner loop
			if (text[i + j] != pattern[j]) break;
			// match found
			if (j == pattern.size()-1) return true;
		}
	}
	return false;
}

This approach is easy to understand and implement but it is too slow since it takes in the worst case O(T*P) time, where T is the length of the text and P the length of the pattern.

The Karp-Rabin algorithm uses a more efficient approach based on hash functions. The idea is to calculate a hash value for the pattern p and for every substring of length P in the text t.

Every time we find a substring with the same hash value of p, we have a candidate that can match the pattern. We can then compare the candidate substring with the pattern character by character to see if they are exactly equal. The reason why we can't say immediately that a substring with the same hash value of t is a match are of course the aforementioned hash collisions or spurious hits.

Notice how this algorithm is very similar to the brute force, but it doesn't always require the second loop that will be necessary only when the hash values of p and the current substring of t are equal.

Of course the approach described so far would be absolutely useless if we were not able to calculate the hash value for every substring of length P in t with just one pass through the entire text. Here is where the concept of rolling hash comes in handy.

bool rabinKarp(string const& text, string const& pattern) {

	const int T = text.size(), P = pattern.size(); 
	
	if (T < P) return false; // no match is possible
	
	const int b = 31, M = 1000000009;

	// calculate the hash value of the pattern
	int hp = 0;	
	for (char c : s) hp = (hp*b + (c-'a'+1)) % M; 

	// calculate the hash value of the first substring of length P
	int ht = 0;
	for (i = 0; i < P; i++) ht = (ht*b + (text[i]-'a'+1)) % M;

	// check character by character if the first substring matches the pattern
	if (ht == hp && text.compare(0, P, pattern)) return true;

	// start the rolling hash        
	for (i = m; i < n; i++) {
		ht = (ht - (text[i - m] * bPow)) % M;
		ht = (ht * B) % M;
		ht = (ht + text[i]) % M;

		if (ht == hp && text.compare(i-m+1, P, pattern)) return true;
	}
	
	return false;
}

With a good polynomial hash function (in terms of b and M parameters) and using the rolling hash technique, the time complexity of the algorithm becomes O(P+T) in average.

Unfortunately, there are still cases when we will need to execute the character by character for each substring of the text (i.e. searching p = “aaa” and t = “aaaaaaaaaaaaaa”).

The way to overcome this problem is the so called “double hash” technique. We compute for every substring two independent hash values based on different numbers b and M. If both are equal, we assume that the strings are identical without even executing the character by character comparison. Also “triple hash” could be teoretically used, but the likelihood of a spurious hit with the double one is so small that this is never necessary in practice.

Conclusion

The rolling hash technique is an important weapon for multi-pattern string search where we have to look at all substrings of fixed length of a given text or we want to find the most frequent substring of a fixed length in a given text.

One of its most known application is the Rabin-Karp algorithm, which can for example be used to detect plagiarism in text files.

Another context wher rolling hashing is extremely useful is data compression. In the LZ family of compressors we need to replace a previously occurred string by an offset to the previous position and a length, codified in several different forms. Using a rolling hash and storing the results and offsets, we don't need to scan all the text multiple time to detect repetitions.

If you liked this post, follow me on Twitter to get more related content daily!

What exactly are containers and what is Docker?

2021-05-02T00:00:00+00:00

Containers are a streamlined way to package and run software applications. They allow to build any application in any programming language using any operating system (OS). Containerized applications can run anywhere on anything and can also be more efficiently deployed into cloud environments.

This post introduces the concept of containers with a particular focus on the Docker platform.

Containers vs Virtual Machines

The concept of container somehow remainds to Virtual Machines (VM), but there are significant differences between them.

A VM is like a single house, where an application lives without sharing resources. Each VM has a full OS which includes its own kernel, file system, network interfaces, and so on. The VM operating system is completely separated from the one of the host, creating an execution environment that overcomes most of the drawbacks of running applications directly on the host OS.

A container is like an apartment in a building, where an application has individual resources but also shares core resources with the other apartments. Each container has a very small OS (i.e. 100 MB for windows nanoserver) and uses the host OS for additional resources. Containers offer an alternative to running applications directly on the host or in a VM that can make the applications faster, more portable, and more scalable.

This is a short summary of the advantages of using containers:

small footprint (no more a full OS to run a single application)
fast startup time and reduced CPU consumption
high portability (all what is needed to run the application is packaged together)
isolation (each container has a slice of OS and memory separated from the one of the other containers)

Docker

Docker is a container technology that is the de facto standard in the industry. Using any system, it is very easy to have Docker installed and running as a service in just a few minutes. The Docker ecosystem includes the following four components:

Image: is set of instructions necessary to create a container. It includes the application dependencies and specifies how to install and set up them.
Container: is an instance of an image that has run, since it may be running, paused, or stopped at the moment.
Docker Engine: is a client-server application with a long-running daemon process acting as a server (dockerd), a command line interface (CLI) client and a set of APIs that can be used to interact with the the Docker daemon. Of course server and client can run on different hosts. The Docker Engine can be considered as an abstraction layer over the host OS and it is essential to run containers.
Docker Hub/Registry: is the place where images are stored and from where are distributed. The Docker Hub is public (currently includes around 5 petabytes of images), so companies usually have private trusted registries in their development environment where they publish/push images as a step of their build pipelines.

As an example, to run a container we could use the following commands in the Docker CLI:

docker search image-name to search an image in the Docker Hub
docker pull image-name to pull the image from the Docker Hub
docker run image-name to run the container

Images and Dockerfile

The goal of containerization is to gather together all the components an application needs to run in a single, contained unit. Docker refers to that unit as a Docker image. Inside the image, there is the application the container is intended to execute and any libraries, configuration files, executables or other components that the application needs. So, an image is a static unit that is stored in a Docker Hub/Registry or in the local file system where the Docker engine is installed as a tarball.

Now the question is: how do we get such an image? An image is essentially built from the instructions used to get a complete and executable version of an application. This set of instructions is defined in a file called Dockerfile.

When we build a Dockerfile, the docker engine executes every single instruction creating an updated version of the image known as layer. In other words, a layer is an intermedaite version of an image and it is uniquely identified by an id (a digest). This concept of layers is particularly handy and enables a building time optimization called layer caching: if you make a change to your Dockerfile, docker will build only the layer that was changed and the ones after that.

So an image is a series of different layers with:

a base image (usually a small part of the OS necessary to execute our container plus some basic dependencies)
a series of read only intermediate layers (each one containing an additional dependency, a configuration step or something else necessary for the application)
an upper writable layer called container (where we usually copy the output of the build of our application).

We can view all the layers that make up the image with the docker history command.

The main commands available inside a Dockerfile are:

FROM: specify the base image used as starting point (or the base OS used to host your system)
RUN: execute the subsequent commands creating a new layer on top of the previous one
ADD: copy new (local or remote) files into the image (for example the directory containing all the binaries after building our application)
ENTRYPOINT: program that is going to run as an executable (i.e. dotnet ourProgram.dll)
EXPOSE: Open container port to the outside

Missing: explain what is a multistage dockerfile (multiple FROM)

Naming and Tagging

When working with container images it becomes important to provide consistent versioning information, so that different image versions can be mantained and retrieved in the registries.

Image tags is the mechanism that provides us with the ability to tag container images properly at the time of building using the syntax docker build -t imagename:tag. If we don't provide a tag, Docker assumes that we meant latest and uses it as a default tag for the image. So latest is just the tag which is applied to an image by default which does not have a tag and it does not necessarily refer to the most recent image version.

It is not a good practice to make images without tagging them and it is important to follow a consistent nomenclature when using tagging to reflect versioning. This is critical because when we start developing and deploying containers into production, we may want to roll back to previous versions in a consistent manner. Not having a well-defined scheme for tagging will make it very difficult particularly when it comes to troubleshooting.

Windows vs Linux

Containers are natively part of the Linux ecosystem, taking advantage of the process isolation and the names spaces to create isolated processes. Neverthless Docker containers are available also for windows. Here the processes are managed in a similar way how are managed users but with a bit more isolation: each container sees its own file system and registry.

It is important to know that it is possible to run windows container only using a windows Docker engine and that when using such engine it necessary to specify if we want to run linux or windows containers. It is not possible to run together Linux and Windows containers.

Containers lifecycle

Docker containers can be started, stopped, paused, killed and they can also go out of memory. The difference between stop and kill is that in the first case the docker engine waits that the cointainer terminates normally, while in the second case it forces the container to terminate immediately. Usually in a production environment we only start the containers and let an orchestrator handle the containers lifecycle.

Docker Hub and Registries

A registry is a stateless, highly scalable server side application that stores Docker images and enables their distribution. It implements the Docker specification defining API to store and serve Docker images. Public Docker images are usually available on a server called Docker Hub, but of course we can also use private registries.

With private registries we can tightly control where our images are being stored, manage our images distribution pipeline and integrate image storage/distribution into our development workflow. Private registry provides also better security over public registry and can be geo-redundant making it faster to download/upload images based on client location.

Some popular private registry implementation are implementations:

Docker private Hub
Azure Container Registry (higly integrated with azure devops pipelines and default choice for windows containers)
OpenShift

The main commands implemented by registries are docker login (login into a private registry providing username and password), docker push (push an image to the registry) and docker pull (pull an image from the registry).

Manage Data

Docker container images are meant to contain reusable applications, but all data saved inside a container doesn't persist when this is removed. This is a design choice whose aim is to keep Docker images small, since large images are not desirable for downloading even in local repositories.

So how does a container maintain data without committing the image each time there are changes to the data? The solution is to mount external storage. In this way the container image doesn’t change and all the state changes are persisted in a storage external to the container.

The following mounting options are available:

tmpfs: share files between the host machine and container so that we can persist data even after the container is stopped. Unlike the others mounting option, the tmpfs has several limitations: 1) the mount is temporary and data does not persist once the container is stopped; 2) data can't be shared between containers; 3) the mount is only available if we run Docker on Linux. The main use of this mounting option is to temporarily store sensitive files that we don’t need to persist in either the host or the container writable layer.
bind mount a file or directory on the host machine into a container, referencing it using the absolute path on the host machine. This mount is very performant, but relies on the host machine’s filesystem having a specific directory structure available. This mean a lower level of portability because if we later run the container on a different host there is no guarantee that the host will have the same directory structure. The main use case of this mount is sharing configuration files between the host and all the containers or sharing source code/build artifacts between the development environment on the host and the container.
volumes: mount a part of the host file system that is directly managed by Docker. This is the Docker native mounting option and is the preferred mechanism for persisting data generated by and used by Docker containers. They have several advantages over bind mounts: 1) easier to back up or migrate 2) volumes can be stored also on remote hosts or cloud providers 3) work on both Linux and Windows containers.

It is also important to know that Docker volumes are an extensible solution through plugins. For example it is possible to use plugins that extends the Docker volumes functionality enabling the mapping of volumes shared among multiple machines.

This is especially useful in case we need to configure multiple replicas of the same service. If we place the configuration data in a network file system, we can make it available to all the containers regardless the host where they are running.

Limit containers resources

By default, a container has no resource constraints and can use as much of a given resource (i.e. memory or CPU cycles) as the host’s kernel scheduler will allow. Docker can enforce two kind of limits on resources usage:

hard limits, allowing the container to use no more than a given amount of a resource;
soft limits, allowing the container to use as much of a resource as it needs unless certain conditions are met, such as when the kernel detects low memory or contention on the host machine.

We can limit the maximum amount of memory/cpu for a container using the -m option (the minimum limit is 4MB) and the maximum amount of CPUs with the option *–cpus=*. For example with the option --cpus="1.5", the container will be able to access at most 1 and half of cpus.

Networking

If we want to run multi-containers applications, we need to have some networking. The Docker networking subsystem is extensible and we can introduce our own network driver if necessary. The following network drivers are typically available for Linux containers:

Bridge: is the default network driver and is the perfect solution for multiple containers to communicate with each other on the same host.
Host: remove the network isolation between containers and docker host, enabling the use of the host network directly.
Overlay: connect multiple Docker daemons on different hosts together so that containers running on these hosts can communicate with each other.
Macvlan: assign a MAC address to a container so that it will appear as a physical device on our network. This is the best choice when dealing with legacy applications that expect to be directly connected to the physical network instead of beeing routed through the docker host network stack.
None: it is used with a custom network driver.

Windows containers netwroking is a little bit different and support the following drivers:

NAT: containers receive an IP address from the user-specified (with the –subnet option) IP prefix. If no IP address is specified, the default 172.16.0.0/12 is used. It is always necessary to change the NAT internal IP prefix, if the host IP is in this same prefix. Container endpoints will be automatically attached to this default network and assigned an IP address from its internal prefix. Port forwarding and mapping from the container host to the container endpoints are supported.
Transparent: containers are directly connected to the physical network. IPs from the physical network can be assigned statically (with the –subnet option) or dynamically using an external DHCP server.
Overlay containers on a swarm cluster can communicate with other containers attached to the same network across multiple container hosts. Each overlay network that is created in a cluster has its own IP subnet, defined by a private IP prefix. The overlay network driver uses VXLAN encapsulation.
L2bridge: containers are in the same IP subnet as the container host. The IP addresses must be assigned statically from the same prefix as the container host. All container endpoints on the host will have the same MAC address due to Layer-2 address translation (MAC re-write).
L2tunnel: driver specific for Microsoft Cloud Stack.

We should choose the network driver which best suits our needs taking into account our physical network infrastructure and our networking requirements (i.e.single vs multi-host). In production environments, an orchestrator networking is often use in place of the native Docker networking.

Docker Compose

Docker compose is a tool that enables us to describe our applications as services within a YAML file (docker-compose.yml). In this context, a service really means a container in production. A service only runs one image, but it encodes the way that image runs: what ports it should use, how many replicas of the container image should run (so that the service has the capacity it needs), and so on. If we want to scale a service, we just increase the number of container instances running the application image in the YAML file, assigning more computing resources to that service.

Compose is particularly handy when working with multi-container applications, since it can guarantee that the containers discover each other in a seamless fashion. Let's consider a quite common scenario where a web application (acting as front-end) calls a backend RESTful web API to fetch some content. The web application needs to access the web API in a consistent fashion. In addition, the web application has a dependency on the web API and that dependency must be expressed when launching the applications in containers. All this is possible thanks to compose and the docker-compose.yml file.

Once the YAML file has been created, we can then use a single command to start and stop all the containers. Docker compose is extremely useful for development environments, automated testing environments and single host deployment. In production environments, orchestrators are usually used in place of compose.

Conclusion

I hope this article helped you understanding the concept of containers and get a general idea about the Docker platform. If you are interested in getting a deeper knowledge of the platform components introduced in this post, I strongly encourage you to go through the documentation available in the official Docker website. Stay tuned for a next post, where I will go through a series of exercises and tasks that will get you more familiar with Docker and the most frequently commands of the Docker CLI.

If you liked this post, follow me on Twitter to get more related content daily!

How to efficiently represent strings using a Trie data structure

2021-03-20T00:00:00+00:00

A trie, or a prefix tree, is a data structure that is used to represent and store a set of words. Although there are many other structures used to store and represent words, the trie is particularly efficient in performing the following operations:

find all the words with a common prefix.
enumerate the words in lexicographical order.

Thanks to these properties, tries have many practical applications like design an autocomplete word system, implement speel checkers or solve word games.

Introduction

A trie is a tree data structure whose nodes store the letters of an alphabet. Words can be retrieved from the structure by traversing down a path of the tree. The term “trie” comes indeed from the word retrieval and it is usually pronounced like “try”, to distinguish it from other “tree” structures.

The basic properties of a trie are:

each path from the root to leaves forms a word;
all the descendants of a node share a common prefix associated to that node.

Each node stores letters as links (or references) to other nodes. There is one link for each possible alphabetic value, so the number of child nodes in a trie depends completely upon the size of the alphabet. The English alphabet has 26 letters, so the total number of children for each node will be 26. Since words share common prefixes, it is somehow necessary to understand where a word in the trie ends. A common solution is to associate to each node a boolean field which specifies whether the node corresponds to the end of a word.

Graphically a trie can be conviniently represented as a tree where each edge is labelled with a single character. So the first letter of each word is represented by an edge connecting the root to one of his children. The end of each word in the trie can be represented with colored nodes that are the endpoint of the edges labelled with their last character.

Trie Node

Each trie node contains the following information:

a boolean variable isWord indicating whether a word ends in this node. It is convenient to initialize this isWord to false since the majority of trie nodes are not representing the end of a word.
a set of links or references to other nodes. Commonly the links can be implemented using a fixed size array having as many element as the alphabet letters. This solution is usually very fast, but have the downside of taking up a lot of memory with empty links. The alternative is to use a map
with characters as the key and the link to the next node as the value. This is also the preferred way in dynamic programming languages like Python.

class Trie {
    
  struct TrieNode {
      
    vector<unique_ptr<TrieNode>> children;
    bool isEndWord;
    
    TrieNode() {
        children.resize(26);
        isEndWord = false;
    }
  };
    
  unique_ptr<TrieNode> root;

  //continue...
};

API Implementation

There are two basic operations provided by a trie: inserting a new word and searching for a given word.

To insert a new word, we need to iterate simultaneously through its characters and the trie nodes. Starting from the root node and the first character of the new word c, we check whether the root contains a valid link corresponding to c. If it is, we can directly move to the node pointed by the link and consider the next character of the new word as well. Otherwise we need also to create a valid link initiating a new node. Once we iterated through all the characters of the given word, we also need to check the boolean flag of the current trie node to true to indicate that word ends there.

void insert(string word) {
        
  auto iter = root.get();
        
  for (char c : word) {
      size_t index = c-'a';
      
      if (!iter->children[index]) {
          iter->children[index].reset(new TrieNode());
      }
      iter = iter->children[index].get();     
  }   
  iter->isEndWord = true;
}

To search a given word, we proceed in a similar way iterating over its characters and moving along nodes and edges of the trie. The difference is of course that we don't create anything. If we try to move along an edge that doesn't exist, we simply return false. Once we iterated through all the characters of the given word, we also need to check the boolean flag of the current trie node to see if an already inserted word ends there. Clearly this last step shall be omitted if we want to verify the existence of a given prefix instead of a word.

bool search(string word) {
        
  auto iter = root.get();
  
  for (char c : word) {           
    size_t index = c-'a';
    if (!iter->children[index]) return false;
    iter = iter->children[index].get();
  }
  return iter->isEndWord;
}

Regardless of the number of words stored in it, the time complexity for both the operation is always O(W) when W is the length of the input word.

Comparison with other data structures

There are other data structures, like balanced trees and hash tables, that give us the possibility to store a dictionary of words and search for a word in it. So when do what are the advantages of using tries? First of all, as we mention during the introduction, such data structures are not efficient in enumerating the words in lexicographical order or in working with a common prefixes. For example a trie allows to efficiently answer to the following questions:

Are there any words that start with a given prefix?
How many words start with a given prefix?
What are all the words that start with a given prefix?

Another reason to use a trie instead of a hash table, is because of potential memory advantages. When several strings have a common prefix, a trie reuses the same nodes and edges for all of them, saving space respect to an hash table that always increases in size. Moreover an hash table needs to deal with hash collisions and this could deteriorate its search time complexity. Of course, in the worst case, when all stored strings start with a different letter, there are no memory optimizations.

Concurrency in C++

2020-01-24T00:00:00+00:00

This post is a summary of how multithreading and concurrency are managed in C++.

Create and Run Threads

Every C++ application has a default main thread represented by the main function. Additional threads can be created instantiating objects of the std::thread class. The prerequisite to create a thread is to include the header. When a thread is created, it usually takes as argument a callback function with the code that will be executed by the thread. The execution of the thread starts just after its creation. The callback can be defined as function pointer, functor, or lambda function. Every instance of the thread class has an associated id. The id of a thread can be recovered through the std::thread::get_id() member function. The std::this_thread::get_id() function allows instead to get the id of the current thread.

New Thread will start just after the creation of new object and will execute the passed callback in parallel to thread that has started it.

#include <iostream>
#include <thread>
#include <mutex>

std::mutex mtx;

void threadFunction()
{
    std::lock_guard<std::mutex> lck (mtx);
    std::cout << "Thread function executing id " << std::this_thread::get_id() << std::endl;
}

class ThreadFunctor
{
public:
    void operator()()
    {
        std::lock_guard<std::mutex> lck (mtx);
        std::cout << "Thread functor executing" << std::endl;
    }
};

int main()
{
    ThreadFunctor f;
    std::thread t1(threadFunction);
    std::cout << "Startin thread function with id " << t1.get_id() << std::endl;
    t1.join();
    std::thread t2(f);
    t2.join();
    std::thread t3([]{
        std::lock_guard<std::mutex> lck (mtx);
        std::cout << "Thread lambda executing"  << std::endl;
    });
    t3.join();
    std::cout << "Exit of Main function"<< std::endl;
    return 0;
}

Handle Threads Lifetime

There are two different ways to handle the lifetime of a thread: std::thread::join() and std::thread::detach(). Here is the difference between the two methods:

join synchronizes the execution of the thread with the execution of its creator. This guarantees that the creatoris blocked until the execution of the created thread is not completed yet. As a consequence, the thread can use all variables int scope of the creator.
detach decouples the execution of the thread from the execution of its creator. Both the creator and the created thread can complete their execution without blocking each other. Due to this fact, it's safe to use in the created thread only global variables. Using variables from the creator scope brings to undefined behaviour. Also using objects with static duration (i.e. std::cout) brings to undefined behaviour when the main thread expires.

After calling join or detach, the std::thread instance has no longer a callback associated to it. (default constructed) will cause the program to terminate.

It is important to always join or detach a std::thread instance with an associated callback, otherwise the std::thread destructor will terminate the program invoking the std::terminate exception. The same behaviour happen calling join or detach on a std::thread instance without an associated callback (i.e. a default constructed thread or a std::thread instance for which join or detach was already called). A strategy to prevent these undesidered terminations is to apply the RAII paradigma (Resource Acquisition is Initialization) creating a wrapper for std::thread that joins automatically at the end of its scope.

#include <iostream>
#include <thread>

class ThreadWrapper
{
    std::thread mThread;
    
    public:
    
        ThreadWrapper (std::thread t): mThread(std::move(t))
        {
            if (!t.joinable()) throw std::logic_error("No associated thread");
        }
	
        ~ThreadWrapper()
        {
            mThread.join();
        }
};

int main()
{
    ThreadWrapper t(std::thread([]{std::cout << "Thread lambda executing"  << std::endl;}));
    std::cout<<"Exit of Main function"<< std::endl;
    return 0;
}

Pass arguments to threads

It is possible to pass any number of arguments to the callback executed by a thread passing additional parameters to the std::thread constructor. By default, all the arguments are passed by value and copied into the thread's stack and any changes made by the thread to the arguments passed does not affect the original arguments. In order to pass an argument by reference, it is necessary to explicitely use the std::ref wrapper from the header.

#include <iostream>
#include <thread>

int main()
{
    int y = 1;
    std::cout<<" Before Thread Starts y = " << y << std::endl;
    std::thread t1([](int& num){num++;}, std::ref(y));
    t1.join();
    std::cout<<" After Thread Joins y = " << y << std::endl;
    return 0;
}

In case the callback associated to the thread is a member function of a class, it is necessary to pass a reference to the class as first argument after the callback. Note that if multiple threads are accessing the member functions of the same object, it is necessary to protect the access to the shared resources of that object.

#include <thread>
#include <iostream>

class Dummy
{
public:
  void DummyMethod()
  {
     std::cout << "Dummy method" << std::endl;
  }
};

int main()
{
  //Execute the LaunchTorpedo() method for a specific Torpedo object on a seperate thread
  Dummy dummy;
  std::thread t1(&Dummy::DummyMethod, &dummy);
  t1.join();

  return 0;
}

Prevent Race Conditions using Atomics and Mutexes

A race condition is a situation where multiple concurrent threads can access shared data and try to change it at the same time. BecauseSince the scheduling algorithm can swap between threads at any time, it is not possible to uniquely determine the order in which the threads will attempt to access the shared data and the final state of those data.

In case of primitive data types (i.e. a bool or a integer counter), the most efficient way to prevent race conditions is to use std:atomic. Atomic types encapsulate values ensuring that access to them do not cause data races. A typical example is when multiple thread execute code incrementing a shared counter.

#include <iostream>
#include <thread>
#include <vector>
#include <atomic>

class CounterWrapper
{
    std::atomic<int> mCounter;
    //int mCounter;
public:
    CounterWrapper() : mCounter(0){}
    int get() { return mCounter; }
    void add(int increment) { mCounter+=increment;}
};


int main()
{
  for(int k = 0; k < 1000; k++)
  {      
    CounterWrapper cnt;
    std::vector<std::thread> threads;
    for(int i = 0; i < 10; ++i) threads.push_back(std::thread(&CounterWrapper::add, &cnt, 10000));
    for(int i = 0; i < 10; i++) threads[i].join();

    if(cnt.get() != 100000) {
       std::cout << "Error at iteration = "<< k << " Counter = " << cnt.get() << std::endl;
    }
  }
  return 0;
}

Without using std::atomic, two threads could try to increment the mCounter variable in parallel giving an incorrect result. Indeed the single line of C++ code incrementing mCounter is actually converted into three machine commands: the first one load the mCounter value into a register, the second one increment the value in the register and the third one update the mCounter value. If one thread load the value before the other one update the value, only one increment will be applied. Atomic types prevent such situation executing the three machine commands as they were only one (the scheduler cannot interrupt these sequence of commands).

A more generic way to fix race conditions is to use mutexes. A mutual exclusion object (or mutex) is an object that allows multiple threads to share the same data, without accessing the data simultaneously. Any thread that needs the data must lock the mutex before using the data in order to prevent other threads from accessing the data. The mutex is set to unlock when the data is no longer needed, so that other threads can access the data. The section of code including the shared data and protected by the mutex is known as critical section.

#include <vector>
#include <mutex>

class CounterWrapper
{
    int mCounter;
    std::mutex mMutex;
public:
    CounterWrapper() : mCounter(0){}
    int get() { return mCounter; }
    void add(int increment) { 
            mMutex.lock();
            mCounter += increment;
            mMutex.unlock();
        }
};


int main()
{
  for(int k = 0; k < 1000; k++)
  {      
    CounterWrapper cnt;
    std::vector<std::thread> threads;
    for(int i = 0; i < 10; ++i) threads.push_back(std::thread(&CounterWrapper::add, &cnt, 10000));
    for(int i = 0; i < 10; i++) threads[i].join();

    if(cnt.get() != 100000) {
       std::cout << "Error at iteration = "<< k << " Counter = " << cnt.get() << std::endl;
    }
  }
  return 0;
}

Instead of using a raw std::mutex a better option is to use std::lock_guard, which is a class template implementing the RAII idiom for mutex. It is essentially a wrapper for the raw std::mutex that locks the mutex in its constructor and unlock the mutex in its destructor.

class CounterWrapper
{
    int mCounter;
    std::mutex mMutex;
public:
    CounterWrapper() : mCounter(0){}
    int get() { return mCounter; }
    void add(int increment) { 
            std::lock_guard<std::mutex> lockGuard(mMutex);
            mCounter += increment;
        }
};

C++17 introduced also std::scoped_lock that is an extension of std::lock_guard but it is able to manage multiple mutexes at the same time.

Synchronize Threads using Condition Variables

Sometimes a thread A needs to wait that a particular condition occurs after a certain task has been completed by another thread B. A naive approach to achieve such synchronization is the "spin waiting" where a global variable is shared between the two threads. Thread A keeps on checking the value of this global variable in a loop until this is updated by thread B. Since the global variable is shared by the two threads it is necessary to use a mutex to prevent race condition in accessing the variable.

#include<iostream>
#include<thread>
#include<mutex>
 
class Application
{
 std::mutex mMutex;
 bool mCondition = false;
public:
 void doSomething()
 {
     std::this_thread::sleep_for(std::chrono::milliseconds(1000));
     std::cout << "Do something " << std::endl;
     std::lock_guard<std::mutex> guard(mMutex);
     mCondition = true;
 }
 void mainTask()
 {
     mMutex.lock();
    
     while(mCondition != true)
     {
        mMutex.unlock();
        mMutex.lock();
     }
    
      mMutex.unlock();
      std::cout << "Do something else" << std::endl;
 }
};
 
int main()
{
      Application app; 
      std::thread t1(&Application::mainTask, &app);
      std::thread t2(&Application::doSomething, &app);
      t1.join();
      t2.join();
      return 0;
}

The main disadvantage of this approach is that thread A will keep on acquiring and release the lock just to check the value of the shared variable. Therefore thread A not only wastes CPU cycles, but also makes Thread B slow, because it needs to lock the same mutex to update the shared variable.

A better way to synchronize the two threads is to use a condition variable. A condition variable is a synchronization primitives enabling threads to wait until a condition occurs. Conceptually, a condition variable is a queue of threads where one or more threads can wait to get signaled that a certain condition has become true, while another thread signals this. A mutex is required along with condition variable to avoid the race condition where a thread checks if the condition has become true at the same time another thread wants to signal that the condition occurred. The combined use of locks and condition variables is called monitor. A monitor has the following workflow:

Thread A calls the wait method on condition variable, which internally acquires the mutex and checks if the required condition is met or not.
If the condition i not met, thread A releases the lock and waits on the condition variable to get signaled. Thread A gets blocked until then and doesn't require CPU cycles.
Thread B signals the Condition Variable when the condition is met
Thread A resumes, acquires the mutex lock again and checks if the condition is actually met or if it is a spurious wake up. In case of a spurious wake up, thread A calls the wait method again. Otherwise it continue its execution.

A std::condition_variable has the following main methods:

wait(lock, condition) It takes as argument the lock and the condition to be met and atomically performs the following operations: release lock, put the thread to sleep until a signal is received or a spurious wake up happen. When the thread wakes up again, it acquires again the lock before returning. Typically the lock is a std::unique_lock object which is a class template that wraps a raw mutex providing the lock/unlock functionality required by a condition variable.
notify_one() It wake up only one of the threads waiting on the condition variable.
notify_all() It wake up all the threads waiting on the condition variable.

#include<iostream>
#include<thread>
#include<mutex>
#include<condition_variable>
 
class Application
{
 std::mutex mMutex;
 std::condition_variable mCondVar;
 bool mCondition = false;
public:
 void doSomething()
 {
     std::this_thread::sleep_for(std::chrono::milliseconds(1000));
     
     std::cout << "Do something " << std::endl;
     
     std::lock_guard<std::mutex> guard(mMutex);
     
     mCondition = true;
     
     mCondVar.notify_one();
 }
 void mainTask()
 {
     std::unique_lock<std::mutex> locker(mMutex);
     
     mCondVar.wait(locker, [this](){return mCondition;});
    
     std::cout << "Do something else" << std::endl;
     
     mCondVar.notify_one();
 }
};
 
int main()
{
      Application app; 
      std::thread t1(&Application::mainTask, &app);
      std::thread t2(&Application::doSomething, &app);
      t1.join();
      t2.join();
      return 0;
}

Returning Values from Threads

The naive way the get values back from a thread is to pass a pointer (or a reference) to the expected value in the thread constructor. The creator of the thread can then wait for the value using a condition variable. When the created thread sets the value and signals the condition variable, the creator will wake up and fetch the data from the pointer.

A simpler solution that does not need any mutex or condition variable is to use two class templates:

std::future stores the value that will be assigned in future by the created thread and provides a get() member function to access that value. If the thread's creator tries to access the value of the future before it is available, the get() function will block it until the value has ben set by the created thread.
std::promise represents a promise to set the value in future. It provides a set_value() member function that is used by the created thread to set the value.

Each std::promise object has an associated std::future object with the same template parameter. The std::future object is generated by the get_future() member function of the std::promise object and will give the value once this has been set by the std::promise object.

#include <thread>
#include <future>
#include <iostream>
 
void getInt(std::promise<int>* promInt)
{
    std::this_thread::sleep_for(std::chrono::seconds(1));    
    promInt->set_value(100);
}
     
int main()
{
    std::promise<int> promisedInt;
    std::future<int> futureInt = promisedInt.get_future();
    std::thread t1(getInt, &promisedInt);
    std::cout << futureInt.get() << std::endl;
    t1.join();
    return 0;
}

The described mechanism is absolutely generic and can be extended the the case where a thread needs to return multiple values at different points of time. The onlt thing to do ispass multiple std::promise objects to the created thread and fetch multiple return values from their associated multiple std::future objects.

Packaged Task and Async

In addition to std::promise, there are two ways to associate a thread to a std::future object: std::packaged_task and std::async.

A std::packaged_task is a class template that wraps a callable element and transfers its result automatically to a std::future object, so that it can be retrieved asynchronously using the std::future get() method. Similarly to std::function, the template parameter of std::packaged_task is the signature of the function, while its constructor takes the function as an argument. Then the get_future() member function creates a future with the same template parameter as the return type of the function. The execution of a packaged_task object starts only when it is invoked in the same thread or it is moved to another thread.

#include <iostream>
#include <thread>
#include <future>
#include <string>
 
int main()
{
    // Create two packaged_tasks that encapsulated a lambda function
    std::packaged_task<int (int)> taskSum([](int x){
        return x+1;
    });
	
     std::packaged_task<int (int)> taskSub([](int x){
        return x-1;
    });
 
    // Fetch the associated futures from packaged_tasks
    std::future<int> resultSum = taskSum.get_future();
    std::future<int> resultSub = taskSub.get_future();

    // Execute taskSub in a separated thread
    std::thread t1(std::move(taskSub), 1);
    t1.join();

    // Execute taskSum in the same thread
    taskSum(1);

    // Fetch the results frome the futures
    int dataSum =  resultSum.get();
    int dataSub =  resultSub.get();

    std::cout <<  dataSum << " " << dataSub << std::endl;
    return 0;
}

A std::async is a function template that takes a function as an argument and returns a std::future, so that the result of the function can be retrieved asynchronously using the std::future get() method. The future has the same template parameter as the return type of the function. The first argument in std::async is optional and allows to define the launch policy that controls how the passed function is executed. There are two different launch policies:

std::launch::async guarantees an asynchronous execution of the passed function in a separate thread
std::launch::deferred guarantees a lazy execution of the passed function in the same thread owning the std::future object (the execution starts only when the std::future get() method is called) If no launch policy is specified, the behaviour is implementation dependent. Some implementations can decide to run the function asynchronously or not depending on the current system load. The other std::async parameterss are the function to be executed and its input arguments.

#include <iostream>
#include <thread>
#include <future>
#include <string>
 
int main()
{
    // create and launch sum computation as a separate thread
    std::future<int> resultSum = std::async(std::launch::async, [](int x){
        return x+1;
    },1);

    // Fetch the results frome the futures
    int dataSum =  resultSum.get();

    std::cout <<  dataSum << std::endl;
    return 0;
}

Both std::asynch and std::packaged_task allows to wrap a function and possibly execute it in a separated thread. Moreover both std::asynch and std::packaged_task sets the function return value to the std::future only at the end of the function execution. Anyway, a std::packaged_task can be considered as a lower level construct for implementing std::async: std::async wraps a std::packaged_task and possibly calls it in a different thread.

To summarize:

the main advantage of std::packaged_task over std::async is to decouple the creation of the std::future from the execution of the function
the main advantage of combining std::promise and std::future over both std::asynch and std::packaged_task is that they give more control over the std::future object. This because the std::promise can set the value to the std::future also at a different time than at the end of the function call.

Franco Fernando

Consistent hashing and rendezvous hashing explained.

Scaling cache servers

Partition data across multiple servers

Problem Generalization

Naive approach

Consistent hashing

Virtual Nodes

Consistent Hashing in Action in Production Systems

Rendezvouz Hashing

Comparison between Rendezvouz and Consistent Hashing

Conclusion

The definitive guide to load balancers.

Introduction

Definition

Hardware and Software Load Balancers

Layer 4 and Layer 7 Load Balancers

Servers selection strategies

Redundancy

Advantages

Clustering

Conclusion

All what developers need to know about proxy servers

What is a proxy?

Forward Proxies

Reverse Proxies

Forward vs Reverse Proxies

API Gateways

Conclusion

Three effective ways to create modulo n integer sequences.

Integer modulo operator

Comparison

Modulo with bitwise and operator

Conclusion

The complete guide to the Disjoint Set Union data structure.

Data Representation

Naive Implementations: quick find and quick union (O(N))

Weighted Union (O(logN))

Path Compression (O(αN\alpha NαN))

Applications

Conclusion

Docker Laboratory

Running a container

Working with the Docker Command Line Interface (CLI)

Building and Running Custom Container Images with Dockerfile

Interact with a running container

Making Changes to a Running Container

Persist Changes to a new Image

Working with multiple containers

Working with Data Volumes

Docker Networking

Working with Docker-Compose

Conclusion

How to efficiently compare strings with rolling hash

Polynomial Hash Function

Modular Arithmetic and Horner's Rule

Rolling Hash

Rabin Karp Algorithm

Conclusion

What exactly are containers and what is Docker?

Containers vs Virtual Machines

Docker

Images and Dockerfile

Naming and Tagging

Windows vs Linux

Containers lifecycle

Docker Hub and Registries

Manage Data

Limit containers resources

Networking

Docker Compose

Conclusion

How to efficiently represent strings using a Trie data structure

Introduction

Trie Node

API Implementation

Comparison with other data structures

Concurrency in C++

Create and Run Threads

Handle Threads Lifetime

Path Compression (O( $\alpha N$ ))