So while the wall jumping worked ok, I added a few more tweaks to it to make it work better. Before it was working by raycasting alone and a simple distance check. Now it takes into account the surface normal so we can account for the angle the player is facing towards the wall.
I also got tired of looking at the blank grey sky, so I threw the clouds skybox from the old forest level back in. Surprisingly it makes the level feel a lot different, but better. Really comes alive now.