I think you should offer them right of first refusal. Do what you are planning on doing, document the labor and costs. When you are ready to sell reach out to them and offer a fair price. Let them make the choice.
IMO this sounds like a family fight where emotions are charged. Offer them an olive branch and by the time you are done their interest will have diminished.