That's a while back for me. Either you blow it up with bombs, or you took a messed up path, was it night or day? If it were day, you may not have been seeing clearly. Also, before you head back to Ordon, you have to go up Death Mountain where you confront a Goron. then you go back to Ordon.