But as long as you are with me, there's no place I rather be
Jess Glynne is saying that it doesn't matter where she is as long as she is with this person. It suggests that all she sees is with this person and it is like a distraction. All she wants is to be together.
Realistically their situation is not ideal and their current relationship will not go any further. Even though he is aware of this, he still wants to try so that he can avoid the pain of leaving.
Relates to marriage as it resembles the phrase 'will you take my hand in marriage'. However the relationship isn't necessarily at this stage yet because Ezra still wants her to 'just say the words.'