Introduction

In this section, the speaker introduces himself and explains his background in data science and Java technology. He also provides an overview of the talk's topic, which is Spring and Redis.

• Brian Stambolten introduces himself as a developer advocate with Redis.

• He talks about his background in Java technology and data science.

• The speaker gives an overview of the talk's topic, which is Spring and Redis.

Vector Similarity Search

In this section, the speaker discusses vector databases, vector search engines, and vector similarity search. He explains how these techniques can be used to deal with unstructured data such as images, video blobs of text, DNA sequences etc.

• The speaker talks about how unstructured data has become more prevalent over time.

• He explains what vector databases are and how they can be used to store highly dimensional data that would be too hard to deal with in a SQL environment or a NoSQL environment.

• The speaker proposes using Redis as a solution for dealing with unstructured data locally.

Redis as a Computing Engine

In this section, the speaker discusses how Redis can be used as a computing engine. He compares it to SQL databases and explains how it can bring computation to where the data is located.

• The speaker compares Redis to SQL databases.

• He talks about resorting to stored procedures in SQL databases for faster processing times.

• The speaker explains how Redis exploits the possibility that the data is already in Redis or that it can be put in Redis to do things with that data locally.

High Expectations for Search Experiences

In this section, the speaker discusses how people have high expectations for search experiences and how natural language searches are becoming more prevalent. He talks about how semantic meaning is important when dealing with multi-dimensional data.

• The speaker talks about how people expect powerful search experiences.

• He explains how natural language searches are becoming more prevalent.

• The speaker discusses the importance of semantic meaning when dealing with multi-dimensional data.

Netflix Recommendations

In this section, the speaker talks about data science and how unstructured data is put into a format that is easy to manipulate in a mathematical matrix operation. The speaker also discusses how vectors are used in machine learning models to find interesting correlations between data.

Vectors

• Vectors are arrays of numbers that represent something in n-dimensional space.

• Machine learning models use vectors to find interesting correlations between data.

• Vectors can represent features of data at different levels of granularity.

• Similarity between vectors can be calculated based on direction and size.

Embedding

• Generating a vector embedding for multi-dimensional data involves choosing an embedding method, cleaning and pre-processing the data, training the embedded model with target data, and generating embeddings.

Introduction to Vision and Language Models

In this section, the speaker introduces the advancements in vision models and language models over the last decade.

Advancements in Vision Models

• Over the last 10 years, vision models have significantly improved in their performance.

• YOLO and other families of vision models can now classify and detect objects with high accuracy.

• Pre-trained AI models are available for use in code today for tasks such as sentiment analysis, chatbots, and audio recognition.

Advancements in Language Models

• Large language models are now capable of generating text that captures meaning within a context.

• Vector similarity has become a simple tool once powerful embeddings are created.

• Pre-trained sentence-based models such as SBERT can be used for specific corpuses of data.

Intermediate Layers and DJL Framework

In this section, the speaker discusses intermediate layers in neural networks and introduces the DJL framework for machine learning.

Intermediate Layers

• Intermediate layers contain packed information that can be used as embeddings.

• These embeddings can be used instead of going all the way out to the output layer.

DJL Framework

• The DJL framework is a Java-based framework for machine learning that abstracts PyTorch, MXNet, TensorFlow, or ONNX script under its covers.

• It provides syntactic sugar to make it easier to work with these frameworks on Java platforms.

Redis as a Multi-Model Memory Store

In this section, the speaker discusses how Redis can be used as a multi-model memory store for data.

Redis as a Computing Engine

• Redis can be molded to handle computing engine tasks.

• Computation is done where the data is stored, which can lead to significant gains.

• SQL databases have stored procedures, but they require client-side computation.

Powerful Search Experience

• 80% of generated data is unstructured and growing over time.

• People expect powerful search experiences that deal with this unstructured data.

• Similarity search and clustering are closely related and useful for dealing with vectorized data in multi-dimensional space.

• High expectations exist for searches that take into account not only actual values but also semantic relationships between features of the data.

Vectorization of Data

• Vectors are used to represent multi-dimensional spaces in which different features of the data correspond to different places in that space.

• Embeddings or vectorizations are used to generate vectors for pieces of multi-dimensional data.

• The steps involved in generating an embedding include choosing an embedding method, cleaning/pre-processing the data, and training the embedded model.

Understanding Multi-Dimensional Data in AI

In this section, the speaker discusses the challenges of understanding multi-dimensional data encoded in deep neural networks.

Vision Models and Explainable AI

• The performance of vision models has significantly improved in the last decade.

• Despite this progress, it is still difficult to understand what happens inside a deep neural network.

• Researchers are working on developing explainable AI, but it remains a challenge.

Language Models for Textual Data

• Large language models are excellent at generating text and creating vectors that can be used to compare previously incomparable things.

• Vector similarity becomes a simple tool once powerful embeddings have been created.

• Previously, vectors were not informative enough, but now they carry more information about the data.

Pre-Trained AI Models

• There is an ever-growing collection of pre-trained AI models that can be used for various tasks such as sentiment analysis, chatbots creation, vision models with audio and more.

• These pre-trained models are trained on specific corpuses of data such as medical or legal data or general Wikipedia-type data.

• It is possible to extend training with your own data or play around with different models to get better results.

DJL Framework for Machine Learning in Java

In this section, the speaker introduces DJL Framework for machine learning in Java.

Introduction to DJL Framework

• DJL Framework has emerged as a framework for machine learning that abstracts PyTorch so you can do it in Java while running PyTorch model under the covers.

• It provides syntactic sugar and does much of the legwork for you.

Vectorizing with Java

• The speaker demonstrates how easy it is to vectorize something using Java.

Knowledge of Redis

In this section, the speaker talks about Redis and its architecture.

Redis Architecture

• Redis stands for Remote Dictionary Server.

• The initial data structure was a named value pair.

• Redis is an in-memory first database that can be optionally persistent.

• Redis has become a Band-Aid for bad architecture, but it's not recommended to use it as such.

• There are various data structures in Redis, including strings, bitmaps, bit fields, hashes, lists, sets, sorted sets, Geo indexes and hyper log lock.

Redis Extensions

• Redis has an extension ecosystem where you can add modules to add new data structures and operations on those data structures.

• RediSearch is a full-blown Lucene-like search engine that allows you to do searches on the Redis side of the fence.

• RediStack is a package that includes five modules: JSON search, graph database (Neo4j-like), time series and Bloom filters.

Vector Similarity in Redis

• Redis is good at storing Blobs of data and can store byte arrays efficiently.

• A vector type field was added to the search engine so similarity searches could be done against Blobs of binary data in hashes or JSON documents.

• Different models can be used for different purposes such as faces or sentiment analysis.

Introduction to Redis

In this section, the speaker introduces Redis and demonstrates how it can be used in a Java application.

Using Redis with Spring Data

• The speaker shows how to use Redis with Spring Data.

• Two JSON documents are stored in the database using Redis.

• Redis on Spring can be used like JPA but is faster.

Vectorizing and Indexing Vectorized Data

In this section, the speaker discusses vectorizing and indexing vectorized data using Redis.

Facial Similarity Search

• The speaker demonstrates facial similarity search using plain Java code.

• Faces are detected and extracted from an image using a retina face detection model.

• The criteria for finding a model in the model sue is discussed.

• A bounding box image is saved, and the first face is extracted from the image.

• The results of the facial similarity search are shown.

Vectorization and Searchable Annotations

In this section, the speaker explains how to create searchable annotations in a Spring model using vectorization. The data comes from a CSV file and images that are mapped to those rows in the CSV file.

Creating Searchable Annotations

• Use the index annotation from Redis on Spring to annotate the field that represents the embedding.

• Use the vectorized annotation to tell the framework to get the image from a URL in the model and create the embedding.

• The demo is a Spring web htmx application where you can upload an image, detect faces, extract them, create an embedding for the input image, and then do vector similarity search against the database.

Index Annotation

• The index annotation has more meta-information about how you want to index insert a specific field.

• For byte arrays, you have to tell it that you want this to be a vector in the search engine schema.

Vectorize Annotation

• The string image resource points to an image somewhere use the resource loader in Spring find me that image turn into a byte array run it through the embedding process and put

the results in the destination field so on the image embedding field below and so that's all you need to do vectorization and be that becoming searchable two annotations in your spring model.

Demo of Vector Similarity Search

In this section, we see a demo of how vector similarity search works.

Running Demo

• Pre-load around 11,000 images.

• The vectorization process happens on save but can also be done in bulk.

Celebrity Match Demo

• The future of this demo is to enable the camera, but for now, you can upload an image and do a facial detection search against the database.

• When you do vector similarity searches, the same image with the same vectorization values will be the first match.

• The speaker takes that image and does facial detection on it, trying to find where faces are. That takes some CPU cycles but then he grabs the face markers and opens them up a little bit so he can get a proper portrait and then he vectorizes that and does the search.

Image Display

In this section, the speaker talks about an image that is being displayed.

The Image

• The speaker mentions that there is an image being displayed.