This popular idiom is commonly used in both social and professional situations, and the literal meaning comes from actually rocking a boat, which could possibly capsize the boat. In the idiomatic sense, it means causing unnecessary problems which can lead to a worse situation.
Further examples:
He really didn’t want to rock the boat with his girlfriend after he upset her last week, so instead, he bought her flowers and brought her on a date.
He rocked the boat with his colleagues after what he said in the canteen. I think a complaint will be made against him.