What you want is fine, but you still might want one shared bank.
I had a setup like you described but changed it by paralleling the batteries in one large bank.
Paralleling the batteries, effectively make one large battery, and connect the inverters/chargers to this large battery could be more...