Yes, the JWT is signed with the secret key of your extension.
Depending on what library you’re using or how you’ve implemented the signature verification, you might need to base64-decode the key prior to using it to verify the JWT.
You also need to use the correct hash algorithm for the verification.
Maybe it helps, here’s where my code does the actual verification of the JWT (PHP based EBS):
https://git.dev.hellcat.net/users/hellcat/repos/ttvext-bbswall/browse/src/AppBundle/Services/JwtHandler.php#54