Basic and Advanced On-Chip Busses and Interconnects

This chapter will begin by providing an overview of the buses and interconnects that are used within an SoC. We will introduce the concepts of data sharing and coherency and how to solve their associated challenges. This will give you a good introduction to the Advance Microcontroller Bus Architecture (AMBA) and the Open Core Protocol (OCP) protocols. Finally, we will cover the data movement within an SoC and how to use DMA engines.

In this chapter, we’re going to cover the following main topics:

• On-chip buses and interconnects overview
• ARM AMBA interconnect protocols suite
• OCP interconnect protocol
• DMA engines and data movements
• Data sharing and coherency challenges

On-chip buses and interconnects overview

FPGA and ASIC-based SoCs are built using multiple components, which are a combination of modules provided as macros by the FPGA vendor, designed in-house usually in RTL, and third-party modules that require a form of licensing to use. These modules are commonly referred to as intellectual properties (IPs). These IPs are connected in a topology specified by the SoC hardware architecture using buses and interconnects. They collaborate, which means they need to interact at runtime to implement a specific set of tasks as part of the system’s overall functionality. There are many levels of functional complexity and features that a given interconnect supports. These are based on a bus protocol specification such as ARM AMBA, OCP, or IBM CoreConnect, to mention a few. In this chapter, we will focus on ARM AMBA, which is a collection of bus protocols grouped under a specific AMBA standard revision. However, we will also cover the OCP bus protocol...

ARM AMBA interconnect protocols suite

AMBA is a collection of bus and interconnect specifications provided by ARM for use in SoCs to attach initiators to targets with different levels of features and protocol complexity. It is an open standard and free to use in SoCs. The specification can be accessed from the ARM website at https://developer.arm.com/architectures/system-architectures/amba/specifications.

The current revision is revision 5. Historically, each newer revision kept backward compatibility with the previous ones but also added new bus protocols with newer features that were developed to keep pace with the modern SoC complexities and higher performance demands. AMBA had to evolve to support multi-core and multi-cluster CPU topologies, which also use some sort of accelerator with all the challenges they present while maintaining the support for lower performance and simpler SoC topologies. The more complex a hardware system becomes. the more power-hungry it will become...

OCP interconnect protocol

OCP is an industry standard that provides the specification of a socket-based bus interface that can be used in an interconnect for modern SoCs. It is developed and maintained by the OCP Working Group. The OCP specification is available from Accellera for free and requires the user to accept the Accellera OCP Specification License. Other materials that are provided with the OCP protocol can be found under the Apache 2.0 license. To download the OCP specification, go to https://www.accellera.org/downloads/standards/ocp/ocp-license-agreement.

This section only provides a brief introduction to the OCP protocol so that you can gain a high-level understanding of its mechanics and characteristics, as well as compare it to AMBA-based bus protocols such as AXI-4 or ACE-4. However, you are encouraged to study the relevant sections of the specification to further your knowledge of the protocol and its implementation details.

OCP protocol overview

The OCP protocol...

DMA engines and data movements

Modern SoCs and high-performance IPs include DMA engines to offload data movement from the CPUs and perform it as desired by the software, with the option to notify the CPU once the operation is completed. There are two categories of DMA engines: DMA engines that are included within an IP, and central DMAs that are standalone and connected to the SoC interconnect like any other SoC IP.

IP-integrated DMA engines overview

IP-integrated DMA engines act as data movers on behalf of the IP and the CPU, so the CPU won’t be needed to copy data from system memory to the IP’s local storage or vice versa. Rather, when programmed and armed, once the data is needed or received, the DMA engine autonomously performs the data transfer from source to destination. Then, its control hardware notifies the CPU when the operation has finished executing, usually via an interrupt. The following diagram illustrates an IP-integrated DMA engine:

...

Data sharing and coherency challenges

Modern SoCs are constructed using multiple data compute engines such as CPUs, GPUs, custom hardware accelerators, and high-performance IP peripherals with integrated DMA engines. They process data that is shared laterally and passed from layer to layer as these different processing engines cooperate. These complex topologies make the system design more challenging in terms of making sure that the data is safe as it is accessed, used, and updated while also ensuring it is coherent when these processing engines make efforts to optimize the data access, such as by using integrated caches. Accessing data concurrently and safely means accessing it atomically and coherently without paying a high penalty in terms of software management and using prohibitive locking mechanisms. We want the system architecture to be lock-free and rely on the hardware to provide the optimal mechanisms to implement these data protection mechanisms.

Data access atomicity...

Summary

In this chapter, we introduced buses and interconnects and their crucial roles in putting modern SoCs together. We explored the defining functional features of buses and interconnects and the background behind them. We also looked at the ARM AMBA standards by revisiting their historical evolution and how SoC design complexity also required an evolving standard that can accommodate and help with designing higher performance and features-rich SoCs. We explored all the relevant bus protocols that make up the AMBA standard, their features and characteristics, and what makes them suitable for a specific connectivity need. We also looked at example implementations to get a feel of how these buses are used in modern SoCs. Then, we covered the OCP standard, its bus characteristics and features, and what makes it appealing for many application domains in the SoC design space. We compared the AMBA standard and the OCP bus protocol and how they can be bridged to accommodate mixed standard...

Questions

Answer the following questions to test your knowledge of this chapter:

1. Describe the differences between a simple bus and a Network-On-Chip. What is a multi-threaded master?
2. Which bus protocols are included in AMBA4? Describe the main characteristics of each.
3. Which states are part of the ACE-4 coherency protocol? Describe each.
4. What are the differences between the OCP bus and the AXI-4 bus? What are the similarities between them?
5. How many types of DMA engines are there? Describe the main differences between them.
6. What is a DMA descriptor? What are its fields?
7. What is a Scatter Gather DMA operation?
8. What is a linked list of descriptors? What is the key element in it?
9. How is cache coherency implemented at the SoC system level?
10. What is atomic data access? How can we implement this between two different CPU clusters?