Let’s do the math. The property is an asset worth the sales price (minus realtors fees) minus what is owed on the mortgage (and on the second loan if it is secured by the property.)
So - let’s throw out some sample numbers and see how the math might work. Adjust by putting in your real numbers.
Assume: Property worth 600k Mortgage of 400k Equity of 200k (minus maybe $40k in realtor fees/repairs etc if sold) Rental income from small unit of $1,000 a month Rental income potential of house $2500 a month
To buy him out (pay him $100k or, if he’ll accept your argument that realtors fees would eat 6% of that if the house was sold, get him to agree to $94k) Get help from a parent to refi - if they could co-sign - and have a mortgage of $500k with a PITI payment on a 30 year fixed mortgage of around $2600 (or more depending on your property taxes ) .
At that payment, you could live in the small rental and pay the mortgage expenses from renting the house, with only repairs and maintenance for your housing costs. OR you could stay in the house and after rental income your cost to live in the house would be $1600 a month plus repairs/maintenance. If you could get a roommate for one bedroom in the house, you could cut your expenses even more.
Now I realize that unless you are in a very low cost area of Hawaii, all these numbers are likely much higher, probably double.
I echo what Gerda has to say about considering living in the small rental if you really want to keep the house. People live in tiny houses, it’s different but doable - and if it
Another alternative, if you really cannot afford to stay, is to sell, and take your share of the equity to move to a much lower cost area. If you stay in the US, don’t forget about health insurance costs in your calculations. My medical assistant just moved to Florida to be near her elderly parents, and bought a nice 2 bedroom 1400 s.f. Condo for about $210k. (Florida isn’t my cup of tea but if you want warm weather and a lower cost of living...)
Plug your numbers into all of this. It may or may not make sense to keep it.