Formals vary a lot by college and by occasion. Some colleges host a formal meal every night, and therefore each meal is less of an 'event'. Others hold formals as infrequently as every fortnight, and so formals turn into popular large gatherings, and are an important dates in a college's social calendar.
How exactly you have to dress for your formal depends on the college hosting it. Some college require formal suits/dresses underneath your Oxford gown. Others just require formal wear, while others just require the gown over normal every-day clothes.