Jungo WinDriver  
Official Documentation
Chapter 4: PCI Express Overview

4.1. Overview

The PCI Express (PCIe) bus architecture (formerly 3GIO or 3rd Generation I/O) was introduced by Intel, in partnership with other leading companies, including IBM, Dell, Compaq, HP and Microsoft, with the intention that it will become the prevailing standard for PC I/O in the years to come.

PCI Express allows for larger bandwidth and higher scalability than the standard PCI 2.2 bus.

The standard PCI 2.2 bus is designed as a single parallel data bus through which all data is routed at a set rate. The bus shares the bandwidth between all connected devices, without the ability to prioritize between devices. The maximum bandwidth for this bus is 132MB/s, which has to be shared among all connected devices.

PCI Express consists of serial, point-to-point wired, individually clocked 'lanes', each lane consisting of two pairs of data lines that can carry data upstream and downstream simultaneously (full-duplex). The bus slots are connected to a switch that controls the data flow on the bus. A connection between a PCI Express device and a PCI Express switch is called a 'link'. Each link is composed of one or more lanes. A link composed of a single lane is called an x1 link; a link composed of two lanes is called an x2 link; etc. PCI Express supports x1, x2, x4, x8, x12, x16, and x32 link widths (lanes). The PCI Express architecture allows for a maximum bandwidth of approximately 500MB/s per lane. Therefore, the maximum potential bandwidth of this bus is 500MB/s for x1, 1,000MB/s for x2, 2,000MB/s for x4, 4,000MB/s for x8, 6,000MB/s for x12, and 8,000MB/s for x16. These values provide a significant improvement over the maximum 132MB/s bandwidth of the standard 32-bit PCI bus. The increased bandwidth support makes PCI Express ideal for the growing number of devices that require high bandwidth, such as hard drive controllers, video streaming devices and networking cards.

The usage of a switch to control the data flow in the PCI Express bus, as explained above, provides an improvement over a shared PCI bus, because each device essentially has direct access to the bus, instead of multiple components having to share the bus. This allows each device to use its full bandwidth capabilities without having to compete for the maximum bandwidth offered by a single shared bus. Adding to this the lanes of traffic that each device has access to in the PCI Express bus, PCI Express truly allows for control of much more bandwidth than previous PCI technologies. In addition, this architecture enables devices to communicate with each other directly (peer-to-peer communication).

In addition, the PCI Express bus topology allows for centralized traffic-routing and resource management, as opposed to the shared bus topology. This enables PCI Express to support quality of service (QoS). The PCI Express switch can prioritize packets, so that real-time streaming packets (i.e., a video stream or an audio stream) can take priority over packets that are not as time critical.

Another main advantage of the PCI Express is that it is cost-efficient to manufacture when compared to PCI and AGP slots or other new I/O bus solutions such as PCI-X.

PCI Express was designed to maintain complete hardware and software compatibility with the existing PCI bus and PCI devices, despite the different architecture of these two buses.

As part of the backward compatibility with the PCI 2.2 bus, legacy PCI 2.2 devices can be plugged into a PCI Express system via a PCI Express-to-PCI bridge, which translates PCI Express packets back into standard PCI 2.2 bus signals. This bridging can occur either on the motherboard or on an external card.

4.2. WinDriver for PCI Express

WinDriver fully supports backward compatibility with the standard PCI features on PCI Express boards. The wide support provided by WinDriver for the standard PCI bus — including a rich set of APIs, code samples and the graphical DriverWizard for hardware debugging and driver code generation — is also applicable to PCI Express devices, which by design are backward compatible with the legacy PCI bus.

You can also use WinDriver's PCI API to easily communicate with PCI devices connected to the PC via PCI Express-to-PCI bridges and switches.

In addition, WinDriver provides you with a set of APIs for easy access to the PCI Express extended configuration space on target platforms that support such access (e.g., Windows, Linux, MacOS).

4.2.1. WinDriver APIs for PCI Express

WinDriver provides two sets of APIs - the Low Level API (WD_xxx()) and the High Level API (WDC_xxx()/WDS_xxx(). The Low Level API is kept for backwards compatibility purposes only and should not be used to write new code. Code using the High Level API is usually about 50% shorter, and therefore is much less error prone as it spares the developer the need to deal with internal WinDriver structures, algorithms and mechanisms. Jungo always recommends upgrading old Low Level code to the High Level API. In order to do so use the following table, or contact our support center for further assistance.

Low Level API High Level API
Previous Function New Function
WD_Open(), WD_Version(), WD_License(), WD_AgentStart() WDC_DriverOpen()
WD_Close(), WD_AgentClose() WDC_DriverClose()
WD_Version() WDC_Version()
WD_Sleep() WDC_Sleep()
WD_DebugAdd() WDC_Err(), WDC_Trace()
PCI Functions
WD_PciScanCards() WDC_PciScanDevices()
WD_PciGetCardInfo() WDC_PciGetDeviceInfo()
WD_PciScanCaps() WDC_PciScanCaps()
WD_CardCleanupSetup() WDC_CardCleanupSetup()
WD_CardRegister() WDC_PciDeviceOpen() / WDC_IsaDeviceOpen()
WD_CardUnregister(), WD_EventUnregister(), WD_IntDisable(), WD_KernelPlugInClose() WDC_PciDeviceClose() / WDC_IsaDeviceClose()
WD_PciConfigDump() WDC_PciReadCfgBySlot(), WDC_PciWriteCfgBySlot(), WDC_PciReadCfg(), WDC_PciWriteCfg()
WD_Transfer() WDC_ReadAddrByAddrDescXXX(), WDC_WriteAddrByAddrDescXXX(), WDC_ReadAddrXXX(), WDC_WriteAddrXXX(), WDC_ReadAddrBlock(), WDC_WriteAddrBlock()
WD_MultiTransfer() WDC_MultiTransfer()
Kernel PlugIn Functions
WD_KernelPlugInOpen() WDC_KernelPlugInOpen()
WD_KernelPlugInCall() WDC_CallKerPlug()
Interrupt / Event Functions
WD_IntEnable(), ThreadStart(), WD_IntWait() WDC_IntEnable()
WD_IntDisable(), ThreadWait() WDC_IntDisable()
WD_EventRegister() EventRegister()
WD_EventUnregister() EventUnregister()
DMA Functions
WD_DMALock() WDC_DMAContigBufLock() / WDC_DMATransactionInit()
WD_DMAUnlock() WDC_DMABufUnlock() / WDC_DMATransactionInit()
WD_DMASyncCpu() WDC_DMASyncCpu()
WD_DMASyncIo() WDC_DMASyncIo()
WD_DMATransactionExecute() WDC_DMATransactionExecute()
WD_DMATransferCompletedAndCheck() WDC_DMATransferCompletedAndCheck()
WD_DMATransactionRelease() WDC_DMATransactionRelease()
WD_DMATransactionUninit() WDC_DMATransactionUninit()
Server API Functions
WD_PciSriovEnable() WDC_PciSriovEnable()
WD_PciSriovDisable() WDC_PciSriovDisable()
WD_PciSriovGetNumVFs() WDC_PciSriovGetNumVFs()
WD_IpcRegister() WDS_IpcRegister()
WD_IpcUnRegister() WDS_IpcUnRegister()
WD_IpcScanProcs() WDS_IpcScanProcs()
WD_IpcSend() WDS_IpcUidUnicast(), WDS_IpcSubGroupMulticast(), WDS_IpcMulticast()
WD_SharedIntEnable() WDS_SharedIntEnable()
WD_SharedIntDisable() WDS_SharedIntDisableGlobal()
WD_KernelBufLock() WDS_SharedBufferAlloc(), WDS_SharedBufferGet()
WD_KernelBufUnlock() WDS_SharedBufferFree()

4.3. The pci_dump and pci_scan Utilities

pci_dump is a sample utility console-mode program that is provided with the WinDriver tool-kit and enables you to dump the contents of the PCI configuration registers into a readable format. It can be found in the WinDriver/util directory.

The source code of this utility is provided as well and can be found at: WinDriver/samples/c/pci_dump/pci_dump.c.

pci_scan is a sample utility console-mode program that is provided with the WinDriver tool-kit and enables you to scan the PCI device that are connected to your computer in a readable format. It can be found in the WinDriver/utildirectory.

The source code of this utility is provided as well and can be found at: WinDriver/samples/c/pci_scan/pci_scan.c.

4.3.1. Dump PCI Configuration Space into a file

To log the pci_dump output into a file (dump.txt), do the following:

  • Open a command prompt window.
  • Run pci_dump as pci_dump > dump.txt.
  • Hit Enter continuously, until the program’s execution is completed.

At the end of this process you wil have a dump.txt file in the WinDriver/util directory (from which pci_dump was run).

The same process can similarly be done with pci_scan output as well.

💡 Recommendation

‍If you have any technical question, we firstly recommend searching the manual. If you cannot find an answer to your question, please send us an e-mail to windr.nosp@m.iver.nosp@m.@jung.nosp@m.o.co.nosp@m.m, or if you are already in contact with someone from our team, just send an e-mail to that person directly. Please note that in order to ensure a quick and effective support our engineers may request clear detailed description of the steps you performed, specify which step failed and what was the exact nature of the failure or erroneous behavior that you encountered (including complete error messages, screenshots, logs).

4.4. FAQ

4.4.1. Enabling legacy PCI configuration space read/write for identifying PCI devices on Windows

In version 6.2.0 of WinDriver, the PCI configuration space read/write method on Windows was upgraded to a more advanced method. On rare occasions, this method fails to identify some PCI devices. To resolve this problem, from WinDriver version 10.3.1 you can revert to the legacy PCI configuration space read/write method by doing the following:

Set the PciCfgRwCompat registry key flag in the WinDriver driver INF file (windrvr<version>.inf(e.g. windrvr1640.inf) / windrvr6.inf in earlier versions)) — to 1:

HKR, Parameters, PciCfgRwCompat, 0x00010001, 1

Beginning with version 11.1.0 of WinDriver, the driver INF file contains a similar line, which sets the PciCfgRwCompat flag to 0 (default), so you only need to modify the value of the flag in the existing line to 1:

Open a command-line prompt and reinstall the driver by running the following commands from the WinDriver\util\wdregdirectory (where “WinDriver” is the path to your WinDriver installation directory):

$ wdreg -inf <path to your device INF file> uninstall
$ wdreg -inf <path to the driver INF file> uninstall
$ wdreg -inf <path to the driver INF file> install
$ wdreg -inf <path to your device INF file> install

4.3.2. How do I access the memory on my PCI card using WinDriver?

First, locate the slot to which your card is connected, using WDC_PciScanDevices().

Then get the card’s information by calling WDC_PciGetDeviceInfo().

This information includes the memory range chosen for the card by the Plug and Play system.

Now call WDC_PciDeviceOpen() to install the memory range and map it into both kernel and user mode virtual address spaces.

You can then either access the memory directly from your user mode application (more efficient), by using the user mode mapping of the physical address — returned by WDC_PciDeviceOpen() in ((PWDC_DEVICE)hDev)->cardReg.Card.Item[i].I.Mem.pUserDirectAddr (where i is the index number of the memory range in the Item array) — or pass the kernel mode mapping of the memory — ((PWDC_DEVICE)hDev)->cardReg.Card.Item[i].I.Mem.pTransAddr — to WD_Transfer() / WD_MultiTransfer(), in order to access the memory in the kernel. Check out the documentation of WDC_DEVICE for more info.

This is demonstrated, for example, in the sample code found in the WinDriver/samples/c/pci_diag directory, and in the diagnostics DriverWizard code that you can generate for your PCI card.

⚠ Attention

‍To access memory directly in the kernel, from within a Kernel PlugIn project, you must use the kernel mode mapping of the physical memory address — returned by WDC_PciDeviceOpen() in (PWDC_DEVICE)hDev)->cardReg.Card.Item[i].I.Mem.pTransAddr — and not the user mode mapping that is used to access the memory directly from your user-mode application.

4.3.3. How do I use WinDriver for PCI Express over Thunderbolt?

Assuming your computer's Thunderbolt adapter is correctly configured and your device is recognized by the operating system, then a PCI Express device connected over Thunderbolt should be recognized by WinDriver as a regular PCI Express device and this should not require special actions on the WinDriver side. Some motherboard chipsets require changes in their BIOS settings to enable Thunderbolt support, consult your vendor's documentation for more info.