So now I tried to clear the cache - switch and clear it right away - the server came up to a load of about 5 and after a while it came back down to below 1.
I then tried to open a product page and the load kicked back to 2+ for a minute - opening the pace took very long. So I wrote a script the waits until the load is below 1 and then starts calling pages from the sitemap - I left it running all night.
In the morning the whole sitemap was completely processed - but - opening each page still takes somewhat around 20 seconds - really no idea why but maybe it has something to do with the header I did with Creative Elements?
I switched back to classic theme (123... done) and the server is back to very good performance!
Guess I'll wait until hummingbird 1.0 or so...
Cheers,
BJF