After reading another post I only had to base64 decode the secret before signing the JWT.
Hope this helps someone else too.