If membership of a tribe is passed down through patrilineal descent, what happens to someone without a Jewish father?

To my knowledge, whilst Judaism itself is passed on matrilineally, membership of a tribe of Israel is passed down patrilineally. As someone with a goyische father, where do I fit in with this? Am I a part of my mother’s tribe or do I not belong to one?