Multiple Search Modalities
With the benefits of deep learning and artificial intelligence, we can encode any kind of data into vectors. This allows us to create a search system that uses any kind of data as a query and returns any kind of data as a search result.
In this chapter, we will introduce the rising topic of the multimodal search problem. You will see different data modalities and how to work with them. You will see how text, images, and audio documents can be transformed into vectors, and how to implement search systems independently of the data modality. You will also see the differences between the concepts of multimodality and cross-modality.
In this chapter, we’re going to cover the following main topics:
- How to represent documents of different data types
- How to encode multimodal documents
- Cross-modal and multimodal searches
By the end of this chapter, you will have a solid understanding of how cross-modal and multimodal searches work...
