Intel® Arria® 10 Avalon® Streaming with SR-IOV IP for PCIe* User Guide

Download PDF
ID 683686
Date 1/11/2022
Public
Document Table of Contents
Document Table of Contents x
1. Datasheet 2. Getting Started with the SR-IOV Design Example 3. Parameter Settings 4. Physical Layout 5. Interfaces and Signal Descriptions 6. Registers 7. Programming and Testing SR-IOV Bridge MSI Interrupts 8. Error Handling 9. IP Core Architecture 10. Design Implementation 11. Debugging 12. Document Revision History A. Transaction Layer Packet (TLP) Header Formats B. Intel® Arria® 10 Avalon-ST with SR-IOV Interface for PCIe Solutions User Guide Archive
1. Datasheet x
1.1. Intel® Arria® 10 Avalon® -ST Interface with SR-IOV for PCI Express* Datasheet 1.2. Release Information 1.3. Device Family Support 1.4. Debug Features 1.5. IP Core Verification 1.6. Performance and Resource Utilization 1.7. Recommended Speed Grades for SR-IOV Interface
1.1. Intel® Arria® 10 Avalon® -ST Interface with SR-IOV for PCI Express* Datasheet x
1.1.1. SR-IOV Features
1.5. IP Core Verification x
1.5.1. Compatibility Testing Environment
2. Getting Started with the SR-IOV Design Example x
2.1. Directory Structure for Intel® Arria® 10 SR-IOV Design Example 2.2. Design Components for the SR-IOV Design Example 2.3. Generating the SR-IOV Design Example 2.4. Compiling and Simulating the Design for SR-IOV
3. Parameter Settings x
3.1. Parameters 3.2. Intel® Arria® 10 Avalon-ST Settings 3.3. Intel® Arria® 10 SR-IOV System Settings 3.4. Base Address Register (BAR) Settings 3.5. SR-IOV Device Identification Registers 3.6. Intel® Arria® 10 Interrupt Capabilities 3.7. Physical Function TLP Processing Hints (TPH) 3.8. Address Translation Services (ATS) 3.9. PCI Express and PCI Capabilities Parameters 3.10. PHY Characteristics 3.11. Example Designs
3.9. PCI Express and PCI Capabilities Parameters x
3.9.1. PCI Express and PCI Capabilities 3.9.2. Error Reporting 3.9.3. Link Capabilities 3.9.4. Slot Capabilities 3.9.5. Power Management
4. Physical Layout x
4.1. Hard IP Block Placement In Intel® Cyclone® 10 GX Devices 4.2. Hard IP Block Placement In Intel® Arria® 10 Devices 4.3. Channel and Pin Placement for the Gen1, Gen2, and Gen3 Data Rates 4.4. Channel Placement and fPLL and ATX PLL Usage for the Gen3 Data Rate 4.5. PCI Express Gen3 Bank Usage Restrictions
5. Interfaces and Signal Descriptions x
5.1. Avalon-ST TX Interface 5.2. Component-Specific Avalon-ST Interface Signals 5.3. Avalon-ST RX Interface 5.4. BAR Hit Signals 5.5. Configuration Status Interface 5.6. Clock Signals 5.7. Function-Level Reset (FLR) Interface 5.8. SR-IOV Interrupt Interface 5.9. Configuration Extension Bus (CEB) Interface 5.10. Implementing MSI-X Interrupts 5.11. Control Shadow Interface 5.12. Local Management Interface (LMI) Signals 5.13. Reset, Status, and Link Training Signals 5.14. Hard IP Reconfiguration Interface 5.15. Serial Data Signals 5.16. Test Signals 5.17. PIPE Interface Signals 5.18. Intel® Arria® 10 Development Kit Conduit Interface
5.9. Configuration Extension Bus (CEB) Interface x
5.9.1. Vital Product Data (VPD) Capability
5.9.1. Vital Product Data (VPD) Capability x
5.9.1.1. Determine the Pointer Address of an External Capability Register 5.9.1.2. VPD Capability Implementation
6. Registers x
6.1. Addresses for Physical and Virtual Functions 6.2. Correspondence between Configuration Space Registers and the PCIe Specification 6.3. PCI and PCI Express Configuration Space Registers 6.4. MSI Registers 6.5. MSI-X Capability Structure 6.6. Power Management Capability Structure 6.7. PCI Express Capability Structure 6.8. Advanced Error Reporting (AER) Enhanced Capability Header Register 6.9. Uncorrectable Error Status Register 6.10. Uncorrectable Error Mask Register 6.11. Uncorrectable Error Severity Register 6.12. Correctable Error Status Register 6.13. Correctable Error Mask Register 6.14. Advanced Error Capabilities and Control Register 6.15. Header Log Registers 0-3 6.16. SR-IOV Virtualization Extended Capabilities Registers 6.17. Virtual Function Registers
6.3. PCI and PCI Express Configuration Space Registers x
6.3.1. Type 0 Configuration Space Registers 6.3.2. PCI and PCI Express Configuration Space Register Content 6.3.3. Interrupt Line and Interrupt Pin Register
6.16. SR-IOV Virtualization Extended Capabilities Registers x
6.16.1. SR-IOV Virtualization Extended Capabilities Registers Address Map 6.16.2. ARI Enhanced Capability Header 6.16.3. SR-IOV Enhanced Capability Registers 6.16.4. Initial VFs and Total VFs Registers 6.16.5. VF Device ID Register 6.16.6. Page Size Registers 6.16.7. VF Base Address Registers (BARs) 0-5 6.16.8. Secondary PCI Express Extended Capability Header 6.16.9. Lane Status Registers 6.16.10. Transaction Processing Hints (TPH) Requester Enhanced Capability Header
7. Programming and Testing SR-IOV Bridge MSI Interrupts x
7.1. Setting Up and Verifying MSI Interrupts 7.2. Masking MSI Interrupts 7.3. Dropping a Pending MSI Interrupt
8. Error Handling x
8.1. Physical Layer Errors 8.2. Data Link Layer Errors 8.3. Transaction Layer Errors 8.4. Error Reporting and Data Poisoning 8.5. Uncorrectable and Correctable Error Status Bits
9. IP Core Architecture x
9.1. PCI Express Protocol Stack 9.2. Data Link Layer 9.3. Physical Layer 9.4. Top-Level Interfaces 9.5. Intel® Arria® 10 Hard IP for PCI Express with Single-Root I/O Virtualization (SR-IOV)
9.4. Top-Level Interfaces x
9.4.1. Avalon-ST Interface 9.4.2. Clocks and Reset 9.4.3. Interrupts 9.4.4. PIPE
10. Design Implementation x
10.1. Making Pin Assignments to Assign I/O Standard to Serial Data Pins 10.2. Recommended Reset Sequence to Avoid Link Training Issues 10.3. SDC Timing Constraints
11. Debugging x
11.1. Setting Up Simulation 11.2. Simulation Fails To Progress Beyond Polling.Active State 11.3. Hardware Bring-Up Issues 11.4. Link Training 11.5. Creating a Signal Tap Debug File to Match Your Design Hierarchy 11.6. Use Third-Party PCIe Analyzer 11.7. BIOS Enumeration Issues
11.1. Setting Up Simulation x
11.1.1. Changing Between Serial and PIPE Simulation 11.1.2. Using the PIPE Interface for Gen1 and Gen2 Variants 11.1.3. Viewing the Important PIPE Interface Signals 11.1.4. Disabling the Scrambler for Gen1 and Gen2 Simulations 11.1.5. Disabling 8B/10B Encoding and Decoding for Gen1 and Gen2 Simulations
12. Document Revision History x
12.1. Document Revision History for the Intel® Arria® 10 Avalon® Streaming with SR-IOV IP for PCIe* User Guide
A. Transaction Layer Packet (TLP) Header Formats x
A.1. TLP Packet Formats without Data Payload A.2. TLP Packet Formats with Data Payload
1. Datasheet
2. Getting Started with the SR-IOV Design Example
3. Parameter Settings
4. Physical Layout
5. Interfaces and Signal Descriptions
6. Registers
7. Programming and Testing SR-IOV Bridge MSI Interrupts
8. Error Handling
9. IP Core Architecture
10. Design Implementation
11. Debugging
12. Document Revision History
A. Transaction Layer Packet (TLP) Header Formats
B. Intel® Arria® 10 Avalon-ST with SR-IOV Interface for PCIe Solutions User Guide Archive

12.1. Document Revision History for the Intel® Arria® 10 Avalon® Streaming with SR-IOV IP for PCIe* User Guide

Date

Version

Changes Made

2022.01.11 20.4 Corrected a bit value in Figure 63. Memory Read Request, 64-Bit Addressing.
2020.12.14 20.4 Added notes to the SR-IOV System Settings section in the Parameter Settings chapter to tie the VF assignment granularity restriction and user capabilities registers implementation recommendation to the appropriate parameters.
2020.11.30 20.3 Removed Tables for the PCI Express data throughput and recommended speed grades for Gen2 x4 mode from the Datasheet chapter because this mode is not supported by the Intel® Arria® 10 Avalon® Streaming with SR-IOV IP for PCI Express.
2020.10.19 20.3

Added a note clarifying the granularity restriction for assigning VFs to the Intel® Arria® 10 SR-IOV System Settings section.

Added a note that includes the recommendation on how to implement user capabilities registers to the Configuration Extension Bus (CEB) Interface section.

2020.10.05 20.3

Added the Configuration Extension Bus (CEB) Interface section to the Interfaces and Signal Descriptions chapter.

Updated the VF configuration space register mapping in the Virtual Function Registers section.

2019.12.19 18.0.1 Updated the signal descriptions and timing diagrams for the Function-Level Reset (FLR) Interface.
2019.09.20 18.0.1 Updated the maximum number of Physical Functions (PFs) supported from four to eight.
2019.04.08 18.0 Removed the duplicate PCI Express* Protocol Stack section.
2018.12.28 18.0 Added the note stating that the non-posted tag pool is shared across all enabled physical functions.
2018.08.13 18.0 Added the step to invoke Vsim to the instructions for running ModelSim simulations.
2017.12.12 17.1 Made the following changes:
  • Corrected Feature Comparison for all Hard IP for PCI Express IP Core table: The Avalon-MM DMA interface does not automatically handle out-of-order completions.
  • Added Enable TX-polarity inversion soft logic parameter.
  • Changed the design example in the Intel® Arria® 10 SR-IOV Quick Start Guide to sriov2_top_target_gen3x8_1pf_4vf.qsys. The previous example is no longer available.
  • Removed reference to the PFs in the description of the Control Shadow interface. This interface is only for VFs.
  • Corrected the description of VF offsets in the VF Offset and Stride Registers table.
2017.05.30 17.0 Made the following changes to the user guide:
  • Removed links to the What assignments do I need for a PCIe Gen1, Gen2 or Gen3 design that targets an Intel® Arria® 10 ES2, ES3 or production device? answer. These assignment are already included in the 17.0 release.
2017.05.15 17.0

Made the following changes to the IP core:

  • Added option soft DFE Controller IP on the PHY tab of the parameter editor to improve BER margin. The default for this option is off because it is typically not required. Short reflective links may benefit from this soft DFE controller IP. This parameter is available only for Gen3 configurations.

Made the following changes to the user guide:

  • Updated PCI Express Gen3 Bank Usage Restrictions status. These restrictions affect all Aria 10 ES and production devices.
  • Added statement that Intel® Arria® 10 devices do not support the Create timing and resource estimates for third-party EDA synthesis tools option on the Generate > Generate HDL menu.
  • Added cross references to data alignment and timing diagrams for the Avalon® -ST interface.
  • Corrected definitions of rx_par_err, tx_par_err, and cfg_par_err. These errors are not recorded in the VSEC register. The VSEC register is not supported.
  • Corrected minor errors and typos.
2017.03.15 17.0 Rebranded as Intel.
2016.10.31 16.1

Made the following changes to the IP core:

  • Added final support for the 128-bit interface to the user application.
  • Added support for the Intel FPGA IP Evaluation Mode.
  • Added licensing requirement.
  • Changed timing model support from preliminary to final for most Intel® Arria® 10 device packages. Exceptions include some military and automotive speed grades with the extended temperature range.

Made the following changes to the user guide:

  • Corrected the number of tags supported in the Feature Comparison for all Hard IP for PCI Express IP Cores table.
  • Removed recommendations about connecting pin_perst. These recommendations do not apply to Arria® 10 devices.
  • Changed the recommended value of test_in[31:0] from 0xa8 to 0x188.
  • Added a note saying that all 4 PFs must be programmed to support a Maximum payload size value greater than 128 bytes.
  • Added -3 to recommended speed grades for the 125 MHz interface.
2016.05.02 16.0 Redesigned the SR-IOV bridge. The 16.0 SR-IOV bridge includes the following changes:
  • Changed number of supported PFs from 2 to 4.
  • Changed number of supported VFs from 128 to 2048.
  • Redesigned BAR matching logic, interrupt generation, and error reporting functions
  • Added support for ATS, TPH, including register definitions
  • Added Control Shadow interface to read the current settings for some of the VF Control Register fields in the PCI and PCI Express Configuration Spaces
  • Removed 128-bit interface to the Application Layer and Gen1 support.
  • Redesigned interface to the Application Layer.
  • Updated address map to reflect the fact that ARI is required, TPH, and ATS registers.
  • Added register definitions for TLP Processing Hints (TPH) and Address
  • Updated bit definitions for the following registers:
    • ARI Enhanced Capability ( 0x100)
    • VF Offset and Stride Registers (0x194) to reflect 4 PFs

Added the following new interfaces:

  • Avalon-ST sideband interface to distinguish VF and PF traffic
  • Function Number and BAR Identification Signals
  • Updated SR-IOV Parameter Settings chapter to include the new parameters necessary to support the new functionality.
  • Updated figures in Physical Layout of Hard IP in Intel® Arria® 10 Devices to include more detail about transceiver banks and channel restrictions.
  • Updated default value of many registers to state that they can be set in Platform Designer.
  • Restored the Transaction Layer Packet (TLP) Header Formats appendix.
  • Added transceiver bank usage placement restrictions for Gen3 ES3 devices.
  • Removed support for -3 speed grade devices.
  • Corrected minor errors and typos.
2015.11.02 15.1 Made the following changes:
  • Improved component GUI that simplifies parameterization. Among the changes is a new single parameter, HIP mode that combines all supported data rates, interface widths and frequencies as a single parameter.
  • Added Development Kit interface signal definitions.
  • Removed Legacy Endpoint from Port type parameter options. The Legacy Endpoint is no longer supported for Intel® Arria® 10 devices.
  • Revised discussion on possible conflict between LMI writes and Host writes to the Configuration Space.
  • Replaced previous Getting Started chapter with a simplified Intel® Arria® 10 SR-IOV Quick Start Guide chapter.
  • Corrected Component Specific TX Credit signal definitions.
  • Added definitions for missing signals and corrected Intel® Arria® 10 PCIe with SR-IOV Signals and Interfaces figure:
    • Removed rx_st_err
    • Added tx_cred_cons_sel to TX credit interface
    • Added Hard IP Status interface signals to figure
  • Fixed minor errors and typos.
2015.06.05 15.0 Added note in Physical Layout of Hard IP in Intel® Arria® 10 Devices to explain Intel® Arria® 10 design constraint that requires that if the lower HIP on one side of the device is configured with a Gen3 x4 or Gen3 x8 IP core, and the upper HIP on the same side of the device is also configured with a Gen3 IP core, then the upper HIP must be configured with a x4 or x8 IP core.
2015.05.04 15.0
  • Added Enable Intel Debug Master Endpoint (ADME) parameter to support optional Native PHY register programming with the Intel System Console.
  • Enhanced descriptions of channel placement, added fPLL placement for Gen1 and Gen2 data rates, and added master CGB location, in Hard IP Block Placement In Intel Arria 10 Devices.
  • Removed Migration and TLP Format appendices, and added new appendix Frequently Asked Questions.
  • Added column for Avalon-ST Interface with SR-IOV variations in Feature Comparison for all Hard IP for PCI Express IP Cores table in Features section.
  • Reorganized sections in Debugging.
  • Updated information in SDC Timing Constraints.
  • Removed erroneous mention of tl_cfg_ctl signal, which is not available in this IP core.
  • Removed list of static example designs from Design Examples. You can derive the list from the installation directory where example designs are available.
  • Fixed minor errors and typos.
2014.12.15 14.1 Initial release.
Level Two Title