Black, in most cases, will NOT go with brown.
Black will go with bold colours, and with tan and khaki in the absence of brown.Interesting, and bold combinations can be made with white.
You might look better contrasting your dark coloured shirt with a light coloured pant/skirt/short.
A black jacket can be worn with black pants or skirt with a light coloured shirt or blouse.
