This plant was found growing in a damp grassy area at Orokonui Ecosanctuary north of Dunedin. I believe that it is an exotic weed. The flowers are < 10mm across. The photo shows leaves along the stem as well as the flower spike.
Hi Euan - I think this is heath milkwort (Polygala serpyllifolia), a weed that's fairly widespread around Dunedin.
Thanks John. I've now found it in Gibbons & Brough - Wild flowers of Britain and Northern Europe..
(JPG format, max 500kB)