We are working our way through the history of microprocessors so we can understand what the latest new features are, what gives a performance boost, and what is just marketing hype. The pipeline is a fundamental feature of microprocessors and is the enabler for several other very important speed-up schemes. Let’s see how pipelines work.

1. The CPU to Memory Interface

The first thing to dig into is how your computer gets instructions and data out of the memory and puts data back. First, the CPU fetches an instruction. That instruction might require a chunk of data, or even two. That means a single instruction might take two or three read cycles to get the instruction and data into the microprocessor.

As mentioned, the microprocessor outputs the address on the address bus, and then reads the instruction. If the instruction calls for data, one or more read cycles take place. All the while, the microprocessor is sitting there waiting for the instruction and data to show up.

After the microprocessor gets all the pieces of the instruction and data, it goes to work. Some instructions may take a few steps, so the memory ends up waiting while the CPU works-lots of stop and go and “hurry up and wait.” Seems like a good way to slow things down, right?

2. Complicated Wiring

The main memory in your computer is made up of RAM (Random Access Memory) chips. The microprocessor outputs the address of the data on an address bus. This is a series of wires on the circuit board with one wire for each of the bits in the address. Even low-end microprocessors have 32 or more address lines, so you can see that buses are complex affairs. Then, there is the data bus with about the same number of wires. That’s 64 copper traces on the circuit board (the wires) between the CPU and the memory. Add to that a handful of control signals to be complete. A 64 bit microprocessor would have twice this number, about a hundred bus lines. It takes time to get all these bus lines moving. This is the biggest bottleneck to speeding up a computer. Everything has to work around the relatively slow speed of the instruction and data buses.

Though early microprocessors operated just like I outlined above, it was in the early days of mainframes that someone figured out a way to put everybody to work 100 percent of the time. Back in 1944, the Colossus Mark II was used by the British to break German codes. It introduced an innovation known as a systemic process; just like the systemic process between your mouth and the other end that is still busy processing your breakfast when you are eating dinner. The data went in one end of the Colossus, and before it came out, more data was put in. The only time the CPU had to wait was for the first instruction and data, and the only time the memory bus was idle was after sending off the last instruction of the program.

4. Indigestion

Microprocessors started out as very simple devices without all this systemic process stuff, but then their instructions grew up very haphazardly. Some instructions were much longer than others and the long instruction could take multiple memory read cycles to fetch. Then, the size and number of the data varied. That made the evenly-paced systemic process break down.