Calculate output layer dimension