You probably already put it in and don't remember it. Head back, climb the block, and look for the Door of Time. Enter it and you will be transported back in time to the Temple of Time.
If you didn't already insert the Master Sword, then there should be a statue blocking the Door of Time. Approach the front of the slot and press A to strike it.