When we say 'at the end of the day', we don't literally mean the time at which the day ends. The idea is to state that something is at the core or the heart of what is happening, or that it is the most important thing. For example, if you were arguing about Climate Change, someone might say, "I understand that there are economic implications, but at the end of the day, the sea levels are rising!" This means that, although the issue of 'economic implications' is important, the core or central truth of the matter are the rising sea levels. What you are trying to do when using this phrase is get someone to focus on the central issue, rather than deflecting to other aspects of the discussion. You will hear this used in lots of contexts, social or professional.