If you want to have an actual private session and avoid having your friends join, you can go into your steam privacy settings and set your friends list viewing to private, the API the game uses to read your friends list seems to rely on it being set to public.
A friend and I recently discovered this to be the solution to me being unable to join his session, it would reject me and tell me it is a private lobby even though we have been steam friends for 17 years, as soon as he changed his friends list viewing to Public, I was able to join him, he set it back to private and I couldn't.
Just figured I would pass along the info if anyone is concerned about being caught cause their friend joined randomly.
Good tip, thanks!