Brunei itself was underwhelming.  The city is not that interesting and can be done in one morning and one evening.  For me it was more interesting  in the National Park and along the river.  Three nights and two days is a long enough tour.  (I did a day trip to Serwak in Malaysia, and that did not go well at all.)  If you are Western, insist on the Radisson for the hotel otherwise you will be put in horrible hotels with disgusting food and tons of noise right on the highway.  I changed hotels 3 times for 4 nights.