8. First time versus repeated execution
A piece of code usually takes much more time the first time it is executed than when it is repeated. The reasons are the following:
- Loading the code from RAM into the cache takes longer time than executing it.
- Any data accessed by the code has to be loaded into the cache, which may take much more time than executing the instructions. When the code is repeated then the data are more likely to be in the cache.
- Jump instructions will not be in the branch target buffer the first time they execute, and therefore are less likely to be predicted correctly. See chapter 22.
- In the PPlain, decoding the code is a bottleneck. If it takes one clock cycle to determine the length of an instruction, then it is not possible to decode two instructions per clock cycle, because the processor doesn't know where the second instruction begins. The PPlain solves this problem by remembering the length of any instruction which has remained in the cache since last time it was executed. As a consequence of this, a set of instructions will not pair in the PPlain the first time they are executed, unless the first of the two instructions is only one byte long. The PMMX, PPro, PII and PIII have no penalty on first time decoding.
For these four reasons, a piece of code inside a loop will generally take much more time the first time it executes than the subsequent times.
If you have a big loop which doesn't fit into the code cache then you will get penalties all the time because it doesn't run from the cache. You should therefore try to reorganize the loop to make it fit into the cache.
If you have very many jumps, calls, and branches inside a loop, then you may get the penalty of branch target buffer misses repeatedly.
Likewise, if a loop repeatedly accesses a data structure too big for the data cache, then you will get the penalty of data cache misses all the time.