I assume that you’ve already dug into the website for the Bluetooth SIG, as there’s a lot of info there. I think the answer is generally yes, though it depends a bit on what “use” means. To get the logo I believe one needs that, though if you use a pre-certified module you can leverage their membership, I believe. Test labs which perform Bluetooth testing can also be a resource to tell you what’s needed.
I worked on a BT product last year. According the research my client did, in order to use the BT logo, or even the word “Bluetooth”, you have to be a member. I recall my client telling me it cost him $12,000.