7. Serialization and Deserialization for backend engineers

What is Serialization and Deserialization?

Overview of Client-Server Communication

• The discussion begins with an introduction to serialization and deserialization, focusing on the interaction between a client (e.g., a browser like Chrome) and a server, which can be local or remote.

• Communication methods between clients and servers include HTTP, REST APIs, gRPC, and WebSockets. The focus will be on traditional HTTP communication for this series.

Client-Side Data Transmission

• The client is described as a JavaScript application (e.g., React, Angular), which sends requests to the server using HTTP methods such as GET or POST.

• An example of an HTTP request structure is provided: it includes the URL, headers, and request body containing data that needs to be sent to the server.

Understanding Data Formats

• Emphasis is placed on how clients (JavaScript apps) and servers (e.g., Rust apps) interpret data differently due to their distinct programming languages and data types.

• A key question arises about how data transmitted over the network can be understood by different systems with varying data formats.

Importance of OSI Model in Data Transmission

• Before diving into serialization concepts, a brief overview of the OSI model is suggested for understanding how different layers communicate during data transmission over the internet.

• It’s noted that while deep knowledge of OSI layers isn't necessary for this video, having a high-level understanding aids in grasping how applications interact across networks.

Protocol Development for Language Agnosticism

• The challenge presented involves creating protocols that allow machines in different locations to send and receive data effectively while being language agnostic.

• A common solution involves establishing standards or rules that both clients and servers agree upon for sending/receiving data. This ensures compatibility despite differences in programming languages.

Understanding Serialization and Deserialization

Introduction to Serialization Standards

• The solution for data transmission involves creating a common serialization standard that both the client and server agree upon, allowing them to send and receive data effectively.

• Serialization and deserialization refer to converting data into a common format for transmission or storage, ensuring compatibility across different systems.

Importance of Language Agnosticism

• The goal of serialization is to make data domain-agnostic, enabling machines from various environments to interpret and respond to the information correctly.

Overview of Technologies in Backend Development

• The backend domain encompasses numerous technologies; thus, it's impractical to master all concepts. Focus will be on widely used technologies.

• HTTP remains the most prevalent method for communication between clients and servers, making it essential knowledge for backend engineers.

Database Choices in Backend Engineering

• Various database types exist: relational (e.g., PostgreSQL, MySQL) and non-relational (e.g., MongoDB). This discussion will focus on PostgreSQL due to its popularity among startups and enterprises.

Popular Serialization Formats

• JSON is highlighted as the most commonly used serialization standard (approximately 80% usage), particularly in client-server communication.

• Other serialization formats include YAML and XML (text-based), while binary formats like Protocol Buffers are also available but less frequently used with HTTP.

Exploring JSON Structure

Characteristics of JSON

• JSON stands for JavaScript Object Notation; it resembles JavaScript objects but is not limited to JavaScript applications. It’s widely utilized in configuration files and API transmissions.

Human Readability of JSON

• One significant advantage of JSON is its human-readable format, which facilitates easier understanding by developers compared to other formats.

Key Rules for JSON Formatting

• Essential rules include:

• Starting with curly braces `` and ending with curly braces.

• Keys must be strings enclosed in double quotes "".

• Values can be strings, numbers, booleans, arrays, or nested objects.

Nested Structures in JSON

• JSON supports nested structures where an object can contain another object or array. For example:

• An address field within a user object can have country as a string value alongside other attributes like phone number as a numeric value.

Summary of Key Points about JSON

• Must start with and end with.

• All keys are strings inside double quotes.

• Values can vary: strings, numbers, booleans, arrays or nested objects maintaining similar characteristics.

Understanding JSON in Client-Server Communication

Introduction to JSON

• JSON (JavaScript Object Notation) is a popular serialization standard used for client-server communication, characterized by key-value pairs enclosed in double quotes.

Demonstrating Request and Response Formats

• The demo focuses on the request and response formats, emphasizing how data is serialized and deserialized during transmission between client and server.

Analyzing the POST Request

• A POST request is made to the API endpoint /books, showcasing the JSON format of the data being sent, which includes keys like ID, title, and author.

OSI Model Context

• The discussion touches on the OSI model, explaining how application layer data gets converted into various formats (data frames, IP packets) before transmission over networks.

Mental Model for Backend Engineers

• Backend engineers should focus on understanding that clients must adhere to standards like JSON while servers interpret this format. The intermediary steps of conversion are not their concern.

Data Transmission Process

• After reaching the server as binary data (0s and 1s), it is converted back into JSON format for processing. This highlights that only the final format matters for backend operations.

Server Response Handling

• Upon receiving a request in JSON format, the server processes it and responds with another JSON object containing relevant information structured similarly to the request.

Rendering Data on Client Side

• Clients can render received data from servers effectively; this process can be done using various programming languages such as JavaScript or PHP.

Serialization and Deserialization Overview

• Serialization refers to converting data into a common format for transmission, while deserialization involves interpreting that data upon receipt. This cycle ensures effective communication between clients and servers.