For those who like circuit diagrams, the sketch below is my first attempt at understanding where all my wires need to go. I've marked the heavy battery cables in orange, and the other wires marked in green are a combination of 50A wires with a couple of lighter 10A wires (e.g. for the shunt sensor connections). I'll try to explain the 'before' and 'after' of this system upgrade just to get my own thoughts in order...
BEFORE
Alternator with built-in charge regulator sends current to the positive terminal of the cranking battery, with a second battery cable running back to the engine/alternator to complete the circuit. Simple!
AFTER
Alternator sends current to cranking battery. When voltage across the cranking battery terminals is higher than a certain level (my charger gives me a choice of 12v or 12.6v, so I'll probably use the latter), the dual-input charger knows to start taking current to put into the lithium house battery (with the engine running this should be at just over 14V). The charger I've picked (I chose BMPro's Miniboost Pro based on a combination of price and features - there are better units out there but they cost a lot more) 'blends' alternator current with solar current, and if the solar is really pumping it favours the solar power.
Meanwhile, all 'in' or 'out' net-flows of power are measured by the battery monitor (this works by measuring tiny voltage changes across the length of the 175A shunt). If I've got this right, an hour of 'in' flow of 10A offset by a 2A draw from my chart-plotter and stereo, say, would be recorded by the battery monitor as an net change of 8Ah in the battery level. The monitor I'm using was $50 from Altronics in Melbourne and can measure flows up to 200A. (I've seen monitors ranging from $50 up to about $400, but as far as I can tell they all work the same way and do the same thing).
One more thing - the thermal breaker in the diagram is just something I found wired into the previous solar charge circuit. There will never be enough current to trip it, so I'm just re-using the component as a handy way to isolate the panels. It has a push-button to break the circuit and a swing down lever that when pushed back into place reconnects everything. With my new set-up I can wire in a maximum of 360W of panels, though for reasons of space I'll probably only have about 200W.
Many 'common earth' wiring set-ups would connect the negative wires at various places, but as all of my wires will be in the same small cupboard I'm literally running all common earth wires to a big, chunky brass bus bar with two 8mm posts that I can bolt all the ring-terminals to. That keeps things simple for the future (i.e. when I've forgotten how all this works) and means only one place to investigate bad earthing connections.
It's taken me months to get my head around all of this. Fingers crossed it works. I'd welcome any thoughts on whether or not it will!