Scary bedtime stories about PCI bus

The general euphoria from the appearance of new processors on the market does not give enough time to look at what they have under the hood. The same sides of them that clickers and pushbuttons usually admire do not at all reflect the real state of affairs on the iron front.



And the clouds, meanwhile, were gathering. Yes, processors are faster. Yes, indeed, there are more cores there. And the last one is just a serious architectural problem. At least for the emerging leader of the race.



Who now remembers such archaism as "Architecture"? Not many people who. The word in the machine tool industry has become almost abusive in IT. But once, in the old days, the book "PCI System Architecture" and other, read to the holes on the screen. And what is there so terrible, in this fragment of the past, scribbled about PCI that can lead us into deep thoughtfulness, full of universal sadness?



A terrible figure - 256. Although, at first glance, it is not very terrible. Yes, that's exactly how many, and there is a maximum limit for the presence of independent PCI devices on the PCI bus. The figure seems really sky-high.



Who, in their right mind and solid memory, would think of having more than 5-10 real ones in the computer, i.e. "Physical" devices? Proceeding from these "logical" premises, the recent leader of the processor race introduced limiting settings into its chipset, which made it possible to forcibly limit the magic number to 128, 64, and you won't believe 32 (!) PCI buses. And it was not just a whim, because gave a serious opportunity to save the system chunk (first 4Gb) of memory, which is still endowed with " magic properties"In relation to the operating system. In fact of the matter is that the physical device has been, until recently, not very much.

But time does not stand still, and here I am holding another 64 nuclear miracle hostile technology , Rome from AMD.



And what Does he want, no REQUIRES, for his minimum functioning? Just a little ... bite off for his 64 cores ... space in 80 PCI buses. Bravo. But, after all, we still have 2/3 of the space? Someone has, and someone has then and there.



first time on nebezdonnost PCI bus I noticed configuring PCI extender, so it has turned out that it hung, for a bunch of P2P Pericom bridge, 119 PCI devices required the same number of individual PCI buses. And this is only behind a single x8 PCI expander. And there were 8 such x8 ports. Here, as if by chance, the " magic" of the first 4Gb of system memory emerges , when the lion's share of those 119 PCI devices requires "modest" 32-64 MB for its initialization during BIOS detection for everyone.



It is good that Intel does not eat PCI space so powerfully. If necessary, but it was still possible to hang the second bunch.



But with Romeand the only such "expansion" deprives us of hope for at least some future of such a weighty "bunch" in this AMD system. No, of course, the client can be offered to switch to a supercomputer architecture, the one that will be made to order and will result in 10 times more expensive. But the above configuration, as you have already noticed, is not for each of us. And it is also developed individually for the client's wishes . But the architectural border of the Wishlist is already noticeably close. I don’t want to build a supercomputer in every mobile tower, solely because of the thirty-year “limitation” of the PCI bus. What is most interesting, but the transition to ARM is not an option at all, since we see the same PCI on it, and all the same 30-year restrictions.



Alas, the situation repeats in detail the trend with memory, is it not enough? Shove-sui more! Is there any PCI space? That to pity him, from him will not lose. Alas, it will already decrease. Already, not that in four or two, you can't cut it, which Intel has dabbled in. Already and in full, for serious players - on the verge. And the use of transparent P2P bridges , alas, only a palliative, in this situation.



All that remains is to save each byte of the bus address. Or dream that the big players will be ripe to create SuperPCIExpress IP6 addresses .



UPD

Dear ikle , shared valuable information on the raised topic in his comment



Duck PCI already has everything for this - the letter S in SBDF - as many as 16 bits - the fourth dimension. Moreover, PCI-E already allows you to use different segments: If the OS is not dense and knows how to understand the ACPI MCFG table, then there is no problem right now.



The problem described in the article is a problem of platform implementation: no one bothers to start several segments on the platform right now, correct the BIOS so that it sends it to ACPI MCFG, where to look for the base address of each segment and everything



And from myself I will add that there is a pci command in UefiShell with the -s Seg parameter which just shows the desired PCI segment. The only sadness is that new segments grow exclusively (as I understand it so far) from additional physicalprocessors. And few people meet with this. Multicore is "not quite right." Or, as an option, the P2P bridge should support this option, which is not popular enough among the masses.



UPD2 Many thanks to

everyone who signaled against errors in the text. Corrected it as soon as I could.