https://www.wired.com/story/10-year-old-face-id-unlocks-mothers-iphone-x/
It seems the Face ID system has some trouble telling family members apart, such as children and siblings if they look similar.
The suggestion to try to mitigate this issue is to either make sure to register your face in good lighting, however it still isn't foolproof, and still can sometimes be unlocked by relatives. It may be best to simply disable the face ID feature if this appears to be a problem.
Detouring slightly, face ID can also be fooled by a mask, but is unlikely to be used unless someone is determined to access your phone:
https://www.theverge.com/2017/11/13/16642690/bkav-iphone-x-faceid-mask