My neighbor's new cow-orker at the local minimart is fucking their boss. Oh yeah, that'll be great for career advancement.
But she's excused from cleaning the restrooms -- my neighbor is stuck with that. Oh yeah, ethnic tensions are involved here, too.
Have you cow-orkers who'll resent your possible relationship with your boss? Is your boss married? Could a relationship, revealed, damage the community standing of you, him, the firm? Do you see (or care) about any long-term prospects?
The heart does what it must. Yes, you could grow together. Is that likely?