The only issue I have with multiple brands, are keeping good batteries. Seems like sticking with one brand tool/battery, the batteries last longer. I have never bought an adapter to try on any other brand. I bought Dewalt set in 2003, drill, circular saw, saws all, and light. Lasted for years, but batteries have been rebuilt and junk again. Brother in law gave me his Dewalt tools, same reason. Bought a new Milwaukee set 4 years ago and think they’re better. Have added grease gun, 1/2” impact, weed eater, and want to get a leaf blower next.
If some things aren’t used often, and you have different brands, seems like the batteries go bad quicker if not used on a regular basis.