New job in Jewish Office

I just accepted a job as an executive assistant in a law office. It isnt explicitly a religiously Jewish office, but my boss is Orthodox Jewish, and I respect that. For reference I live in NY,, outside of NYC, and I am Catholic. My hours are M-F 8-4. I was interviewed by my boss and the person I took over for, who was also Catholic and worked there for 3 years. The previous employee told me that on Fridays everyone left around 2:00 PM due to the Sabbath, which I’m familiar with due to where I live. After two weeks there my boss mentioned to me that he noticed I had left at 2 when I was supposed to leave at 4. I responded that I was told by my training that was when everyone left, and that is what I observed first hand when I would leave on Fridays. He just emailed me back saying it was no problem that I was told that, but made no correction that he expected me to stay until 4. So today was the first Friday since that conversation. He happened to be there until 3:30 so I had no issue staying with him to be support if he needed, but when the other employee left at 3, then he left at 3:30 and just said bye and did not say I could also go, I felt that there was a discrepancy. Is it wrong that I feel that just because I’m Catholic/Christian, a Friday mandate to leave early, for whatever the case may be (religious or otherwise) should be extended to all employees if we are all going to be paid out our full salaries (I submit payroll so I know there is no doc from pay on Fridays from those who leave early for religious purposes) or else that is considered unfair or discrimination. Before you answer, imagine the reverse.