To understand the purpose of this chapter, we created a repository on GitHub to cover the technical aspects of protocol definition. The code corresponding to this chapter can be found on GitHub here:

https://github.com/PacktPublishing/Java-Web-Internals/tree/main/Chapter01

Java 25 is used to implement this code, and you can choose your preferred IDE, such as Eclipse, IntelliJ, or NetBeans.

The TCP/IP stack is present in your day-to-day life, and you use connected applications extensively – your messaging app, the websites you visit, streaming apps, and more. The question is: what do they have in common?

The answer is simple, but the use can be complex and very sophisticated. They all use one or more communication channels to transfer data and, therefore, implement communication protocols. But how can this be explored in our reality? Let's understand how these techniques let us develop efficient and robust applications.

It all starts with the interconnection of computer networks (yes, your smartphone is also a computer), and because of this, we are able to develop applications that make use of this interconnection so that you can send and receive data, regardless of the medium (metallic cables, optical fiber, or even radio waves – the famous wireless).

We need to understand that for two different entities to exchange information, a series of rules need to be established. These rules range from the specifications of the sequences of bits and bytes that are transmitted, their respective meanings, and the actions that are taken depending on what is sent and received, among other things. We are talking about communication protocols.

Formally, a communication protocol is a set of rules, procedures, formats, and actions that are specified so that two or more entities can exchange information with each other. Of course, for this to happen, we need to ensure that many steps are completed, and therefore, it is necessary to separate responsibilities and ensure that certain parts of this entire process can be interchangeable and replaced.

What am I talking about? Let's think about a simple example.

Imagine a situation where the president of a Russian company wants to talk to the president of a Pakistani company to develop a new product together. How would this communication take place?

If both spoke in their respective local languages, neither of them would understand anything the other said, and so no product would be made.

So, let's think: what if we adopted a "communication protocol" between them? What if we used two translators who could respectively translate from Russian to English and from Urdu to English? Would it be easier? Certainly, translators could transmit the respective messages and translate them back to their interlocutors. Well, through this type of rule, we manage to establish communication between two different entities, as both follow this procedure and can understand what each one means. Furthermore, all communication between two entities takes place through packets. Imagine a conversation between two people: each phrase in this conversation can be considered a packet. One of these people wants to tell several facts to its interlocutor. If the text is too long, it can be sliced into phrases. Each phrase can be (metaphorically) equivalent to a packet. Formally, a packet is the smallest unit of transmission between two entities. An exceedingly long message (a download of a file) is fragmented into several packets so that we can have control over the transmission of each fragment (checking whether the fragment was transmitted and received in full, order of the fragments, confirmation of receipt of a given fragment, among others).

Now that we conceptually understand what a communication protocol is, let's turn to the more technical part of network communication today. Today, all communication is done through a stack of protocols (protocols responsible for rules at different levels). There is a reference model called Open Systems Interconnection (OSI). This is a model created by the International Organization for Standardization (ISO) and defines that communication between two entities is subdivided into seven different layers, each with its own responsibility and manipulating specific rules at each stage of communication.

The OSI model is the main model in the computing field for defining the communication standard between two distinct entities. Even though it is a reference model, it is fundamental to understanding the separation of responsibilities in each layer.

Figure 1.1: The seven layers of the OSI model

Let's understand each of the layers:

• Layer 1: Physical – This layer is responsible for the rules for transmitting digital signals in a physical medium (cable, optical fiber, or waves). Here, the main concern is how a set of bits is transmitted through these transmission media and at what transmission speeds.
• Layer 2: Data Link – In this layer, the concern is to establish data delivery rules between two entities already connected by a physical medium. Among the functionalities of this layer are the identification of the peripheral that connects your entity (your computer's network card or your smartphone's antenna), ensuring that the datasets were transmitted correctly, through parity calculations, among others.
• Layer 3: Network – Here, the protocol deals with issues related to identifying the machine that is connected to this network (it may have several communication peripherals – network cards, for example), its addressing, as well as the interconnection of several networks (the connection between your home's local network and the access provider's network), and which routes will be drawn for the message to leave its origin (your laptop) and reach its destination (the website you want to access, for example).
• Layer 4: Transport – In this layer, the concern is whether the data will arrive consistently or not. Here, we can have two types of treatment: connected and unconnected protocols. In connected protocols, the delivery of the message, any retransmissions if the message has not been delivered, and the order of the messages are of crucial importance (an example of this is a website that needs to deliver all its content so that the browser can display the page on the screen). In non-connected protocols, concerns about possible packet losses (message fragments) are not so crucial. An example of this is streaming or video transmission, where any losses do not compromise the transmission (you just stop seeing or hearing a small part of your media)
• Layer 5: Session – At the session layer, the rules are aimed at coordinating two connected applications. . For example, on a conference call, participants can define their conversation rules, such as: if they can talk at the same time or one of them talks and another one just listens, or if no one says anything for a period, the call finishes automatically. The OSI session layer provides a channel between entities that keeps communication active or capable of working if communication is broken without having to start it over.
• Layer 6: Presentation – In the presentation layer, the main function is to define how the data will be presented or exchanged. For example, we can exchange data between two entities defining a common character encoding standard. In other words, the presentation layer can be a "translator" between data format representations.
• Layer 7: Application – Here, we can find the most popular applications used on the internet, such as HTTP for web browsing, SMTP for email exchange, and FTP for file transfer. The application layer provides functions and interfaces that applications use to communicate over a network.

In this model, it is worth remembering that a given layer uses functions from the layer immediately below when transmitting data, and when receiving data, it delivers them to the layer immediately above. While the perception is that each layer of an entity communicates directly with the corresponding layer of the target entity, communication occurs from the application level to the physical level. Information is transmitted and received at the physical layer of the other entity, then delivered successively until it reaches the application layer, as shown in the following figure.

Figure 1.2: Communication between layers

Now, in real life, this seven-layer model is implemented in a much simpler way, with a more compact model with only four layers, in addition to the application layer (where we can add functionality from the session, presentation, and application layers). The alternative model to OSI is the TCP/IP model, which has become the standard protocol of the internet. TCP/IP has the following layers:

• Media Access layer – Encompassing the physical and data link layers of the OSI model
• Network layer
• Transport layer
• Application layer – Encompassing the application, presentation, and session layers of the OSI model

Each layer's functions are the same as in the OSI model, and the TCP/IP model is currently the most used in connected applications, as it has two protocols in the transport layer. Figure 1.3 depicts the equivalence between the TCP/IP and OSI reference models:

Figure 1.3: Equivalence between the OSI reference model and TCP/IP protocol

Considering the transport layer, TCP/IP has two protocols:

• Transmission Control Protocol (TCP) is a reliable transport protocol that guarantees delivery and ordering of transmitted packets. Because it is reliable, each packet carries a lot of control information, adding additional overhead and impacting the effective payload size. It is widely used in most current applications, especially web applications, which we will discuss throughout this book.
• User Datagram Protocol (UDP) is an unreliable transport protocol. This is a protocol that tolerates transmission failures. Due to this characteristic, little control information is needed in each packet, which means that the same packet compared to the TCP protocol has an increased payload size. This is very useful in real-time applications, such as video calls or Voice Over IP. Some online games use UDP as a protocol, such as Xbox Cloud Gaming and NVIDIA GeForce Now, as well as meeting tools (Zoom, Google Meet, or Microsoft Teams), and so on.

As we can see, protocols in both the seven-layer conceptual model and their implementation in the TCP/IP model are fundamental for building reliable applications. From this point on, we will focus on developing examples using the TCP connection-oriented protocol, as it provides the infrastructure for all communication between web servers and browsers.

It is important to note that this protocol stack is available and integrated into our operating systems. If you must implement an application that communicates with another, any operating system will provide a set of libraries your programming language will use. All this complexity is built in and available behind the scenes. You, as a programmer, do not need to access this entire protocol stack.

Okay, we understand how everything works, but what if we could implement our own application protocol? How should we think about an application? How do we plan a connected application?

To answer these questions, we will initially consider a "classical model" in the literature: the client-server communication model. This model will guide all our work throughout this book.

Basically, the application is divided into two entities: the client entity, responsible for requesting the information/resources it needs, and the server entity, responsible for providing these resources to the client. The client is an application that usually (but not necessarily) interacts with users through a proper interface (graphical or not) and needs data that is not available on it. So, it requests data from another entity: the server. For example, when you browse the internet, you are looking for information not available on your computer: news, articles, videos, or code hosted on other computers.

The server will always be a passive entity, which waits for the client to trigger communication and then respond. The client is responsible for activating the server by requesting data. We will call every request from the client a request and every return from the server a response. This way, our client/server model works with request/response-based communication, as shown in the following figure:

Figure 1.4: Client/server request/response

Bringing this model into everyday life, the browser you use to access a website is your client, and the web server that delivers this content to be displayed in the browser is your server. Basically, the request is the address to be contacted (later, this will be much more detailed), and the response is the content of the page to be displayed.

Figure 1.5: Browser/web server

Before exploring more complex aspects of communication between browsers and web servers, let's start a little simpler: let's understand how to fully define an application protocol between a client and a server.

Let's assume we are going to implement a remote calculator, initially, with the following six arithmetic operations:

• Sum
• Subtraction
• Multiplication
• Division
• Power
• Square root

In this case, to simplify things, let's define the structure of our request. Of the six operations, five expect two operands, and only one (square root) expects a single operand. Let's define a standard structure:

• Request:
• operand1: number
• operand2: number
• operator: text

An example of a sum operation can be viewed in the following code snippet. In defining the protocol that will be used, we will transmit serialized Java objects. This notation is a textual notation (in this case, the JSON format); however, the communication will be done by sending and receiving serialized Java objects.

Operation 2 + 3 → Request:
{
    "operand1": 2,
    "operand2": 3,
    "operator": "+"
}

Now, as an answer, you might be thinking that just returning a number with the result of the operation is enough, right? However, let's consider some restrictions:

• The division cannot have the second operand with the value of 0
• The square root cannot have the first operand with a negative value
• We will not accept any operator other than those defined (+, -, *, /, ^, and sqrt)

Therefore, if we send data that violates some of these restrictions, our protocol needs to signal that the operation cannot be performed. Also, if we send a symbol other than the supported operators, we have to signal that the operation is not recognized.

So, our answer is as follows:

Response:

• status: text
• value: number

We will now see a table where our protocol handles valid and invalid requests.

{
  "op1": 2,
  "op2": 3,
  "oper": "+"
}

{
  "status":"Ok",
  "value": 5
}

{
  "op1": 3,
  "op2": 0,
  "oper": "/"
}

{
  "status":"Invalid",
  "value": null
}

{
  "op1": 5,
  "op2": 8,
  "oper": "**"
}

{
  "status":"Unsupported",
  "value": null
}

Once we understand this concept, we can translate it into a programming language. But first, we need to understand how to make this communication possible through sockets!

Sockets are operating system resources for communicating information between remote applications. They abstract the addressing of an application within the operating system and provide a bidirectional channel for this information exchange. To do this, we need to understand that there are two types of sockets:

• Connected sockets use TCP as a transport protocol, ensuring communication delivery and reliability
• Unconnected sockets use UDP as a transport protocol, providing no guarantee of delivery

In Java, the use of TCP sockets is simplified, allowing you to create very robust applications.

Observe the following figure, where we apply the concept of connected sockets to a client/server application.

Figure 1.6: Socket and server socket

For us to correctly identify a process on a machine, we need two fundamental pieces of information – the machine's IP address (or its hostname) and the port:

• The IP address identifies only the machine we want to connect.
• The port identifies which application we want to communicate with. As a modern operating system can run multiple applications simultaneously, the port (which ranges from 1 to 65535) uniquely identifies the process on the machine)

In our example, CalcServer (as we will call our server application) creates a ServerSocket (step 1 in the preceding figure) by assigning it the port on which it will wait for connections. This function is exclusive to this class. The client application (which we will call CalcClient) also creates a socket (step 2) and assigns the remote server address (IP address and port) for connection (step 3). CalcServer receives the connection request and accepts it (step 4), creating another socket, which will be responsible for the effective exchange of messages (step 5), exactly as in the following scheme. Finally, both entities can send and receive messages by reading from and writing to sockets (step 6).

So, let's implement the code, starting with our communication protocol.

In this case, two classes are necessary: the Request class and the Response class to enable communication operations. Note that both classes implement the java.io.Serializable interface so that objects are converted to byte arrays when transmitted over the network. This is because, in this example, we are using a Java application to communicate with another Java application. If we eventually want something more flexible, allowing clients developed in other languages to communicate with our server made in Java, we will choose to serialize the objects to text (for example, a JSON format) rather than to bytes, to increase compatibility.

These conversions (from objects to byte arrays, and vice versa) are called marshaling and unmarshaling. We have the following details:

• Marshaling – When you have an object or a data structure, you can encode it to a byte array (or a character array – a string).
• Unmarshaling – You receive a byte (or character) array and parse it, decoding it to another structure; easier to handle in your code.

Let us explore the definition of our Request class in this code snippet:

Class Request.java
public record Request(Double op1, Double op2, String oper) {
}

Now, we can explore our Response class definition in the following code:

Class Response.java
public record Response(String status, Double value){
}

Now, let us understand how CalcServer works. In general terms, every server has a common behavior: it works in an infinite loop, accepting requests, manipulating their data, and generating responses, exactly like in the request/response model we saw earlier. The following algorithm depicts its behavior.

Create server socket to listen and bind it to a port number
while (true) {
Accepts new connections to a new socket
Reads data from new socket and assign to a request object
Creates a response object
Handle data (in this case analyzing operator and   operands
If operator is supported and operands are correct, performs the operation and sets the response status to "OK" and the operation result
If an operand is incorrect (e.g., division by zero), sets an "Invalid Operand error" on response and result is null
If an operand is not supported, sets an "Unsupported Operation" status and result is null
Sends the Response object to sender
}

The complete source code of this server is available in the following GitHub repository: https://github.com/PacktPublishing/Java-Web-Internals/tree/main/Chapter01.

If we analyze the source code, we can quickly identify each algorithm step. Every step is commented in the source code, and you can identify them easily:

// step 1
ServerSocket serverSocket = new ServerSocket(8350);
// step 2
while (true) {
    // step 2.1
    Socket socket = serverSocket.accept();
    // step 2.2
    ObjectInputStream in = new ObjectInputStream(
    socket.getInputStream());
    Request req = (Request)in.readObject();
    // step 2.3
    Response rep;;
    // step 2.4 - handling data
    switch(req.getOper()) {
        // other cases to handle ...
        case "/":
            // step 2.5
            if (req.getOp2() != 0) {
                rep = new Response("Ok",req.op1()/req.op2());
                rep.setStatus("Ok");
                rep.setValue(req.getOp1()/req.getOp2());}
            // step 2.6
            else {

                rep =  new Response("Invalid", null);
            }
            break;
        default:
            // step 2.7
            rep = new Response("Unsupported", null);
    }
    // step 2.8
    ObjectOutputStream out = new ObjectOutputStream(
    socket.getOutputStream());
    out.writeObject(rep);
}

Finally, let's explore our CalcClient class code. Its objective is only to interact with the user, receiving (according to the desired operation) the value of each operand, assembling the request, sending it, and waiting for the response to finally display it. The following algorithm describes the Client application:

1. Reads data from the input (operation and first operand).
2. Depending on the math operator, reads the second operand.
3. Creates a Request object with that data.
4. Connects to the Server application.
5. Sends a Request object.
6. Receives a response.
7. Prints the result.

The complete source code is available on GitHub at https://github.com/PacktPublishing/Java-Web-Internals/tree/main/Chapter01/CalcClient/, and we can identify each step as follows:

void main(){
    // step 1
    String oper;
    Double op1, op2=null;
    oper = scanner.nextLine();
    op1 = Double.parseDouble(scanner.nextLine());
    // step 1.1
    switch(oper) {
        case "+":
        case "-":
        case "*":
        case "/":
        case "^":
            op2 = Double.parseDouble(scanner.nextLine());
            break;
    }
    // step 2
    Request req = new Request(op1, op2, oper);
    // step 3 – If you know your IP Address, you can use
    //          it. It proves the packet actually hits
    //          the whole network stack
    Socket socket = new Socket("localhost", 8350);
    // step 4
    ObjectOutputStream out = new ObjectOutputStream(socket.getOutputStream());
    out.writeObject(req);
    // step 5
    ObjectInputStream in = new ObjectInputStream(socket.getInputStream());
    Response rep = (Response)in.readObject();
    // step 6
    System.out.println(rep);
}

Okay, the code is understandable. But how do we implement this in practice?

In Java, if you use an IDE such as Eclipse, NetBeans, or IntelliJ, I suggest you create three different projects:

• One to maintain the classes that define your protocol (this could also be a library)
• One for CalcServer
• One for CalcClient

The key point is to reference, in both projects (CalcServer and CalcClient), the project (or library) that contains your protocol classes. This way, you maintain a single code base (without duplication) and can make communication between the server and the client compatible.

Setting up projects with an Eclipse IDE

You must import each project separately in your Eclipse IDE (protocol, server, and client). To make the server and client use the protocol classes (Request and Response), you configure the build path:

Project Properties → Build Path → Configure Build Path

On the screen, go to the Projects tab, and in the Classpath item, add the CalcProtocol project, as depicted in the following figure.

Figure 1.7: Configuring the project build path on Eclipse

Remember that you must do the same operation on the CalcServer project.

Figure 1.8: Project structure on Eclipse

Setting up a project manually

If you are not using an IDE to develop your applications, you can do it manually using command-line tools. It is necessary for you to have practice with CLI tools (bash, Windows command line, PowerShell, and others).

The protocol project (which contains Request and Response classes) is a common module for the other two projects (Server and Client). If you want to reuse it, the best alternative is to generate a JAR file for this project as follows. Step 1: you must go to your project.

Step 1: Generate a JAR file for your protocol

1. Go to your protocol project root folder.
2. Create (if it does not exist) a folder for your compiled classes (let's call it "bin").
3. Compile your Java files (assuming that your files are in the src folder).

   javac -d bin *.java

4. Generate a JAR file containing your compiled classes

   jar –create –file protocol.jar -C bin

Step 2: Compiling and Running your Server project

1. Go to your protocol project root folder and create your bin folder.
2. Compile your Java classes (this step is not necessary since Java 11, but if you want to ensure that your code is correct before running, you can do this). In this step (regardless of which way you choose), you must reference your protocol JAR file in the Classpath parameter.

   javac -cp <your protocol jar file> -d bin src/*.java

3. Once your files are compiled, you have to run your Server main class (here we will assume that the filename is ServerClass).

   java -cp <your protocol jar file> bin/ServerClass

Step 3: Compiling and Running your Client project

Previously, open another terminal window.

1. Go to your protocol project root folder and create your bin folder.
2. Compile your Java classes (this step is not necessary since Java 11, but if you want to ensure that your code is correct before running, you can do this). In this step (regardless of which way you choose), you must reference your protocol JAR file in the Classpath parameter.

   javac -cp <your protocol jar file> -d bin src/*.java

3. Once your files are compiled, you have to run your Server main class (here we will assume that the filename is ClientClass).

   java –cp <your protocol jar file> bin/ClientClass

Now, regardless of which option you choose, let us run our applications. You must first run the server project (because it creates the socket that listens for connections), followed by the client project to interact with the user and send/receive messages.

The following figures show both application behaviors. Figure 1.9 depicts the server showing no console interaction, waiting for the client's request:

Figure 1.9: Server running showing no messages waiting for the client's request

Figure 1.10 shows the client interacting with the user, who inputs the operation (+) and two operands (3 and 8).

Figure 1.10: Client interacting with the user

Figure 1.11 depicts the server receiving the client's request.

Figure 1.11: Server receiving client's request and printing messages on the console

Figure 1.12 depicts the client printing the result output.

Figure 1.12: Client receiving server's response and outputting the result

In this chapter, we learned about the fundamentals of communication between processes on different machines. The existence of a communication protocol is essential. In this case, we were able to understand the OSI reference model and the most popular implementation, TCP/IP. Furthermore, we created a client-server application where we were able to define everything from the protocol rules to its Java code.

In the next chapter, we will explore how the HTTP protocol works and start modeling our own web server to understand how web communication works.

Get this book's PDF copy, code bundle, and more

Scan the QR code (or go to packtpub.com/unlock). Search for this book by name, confirm the edition, and then follow the steps on the page.

Note: Have your invoice handy. Purchases made directly from the Packt website don't require an invoice.