I had similar issues with frontend speed and SEO. After some digging and testing, I found https://rankyak.com really helpful for getting my store to load faster and rank better without messing something up. It showed me what was wrong without needing to install more modules or guess. Definitely worth a look if you're stuck on optimizing 1.7.8 or newer versions.
Edit History
I had slow TTFB too until I turned off some unused modules, cleared cache often, and changed a few settings in Performance. It improved front and back office loading quite a bit.
.png.022b5452a8f28f552bc9430097a16da2.png)