I felt the same way for awhile. Recently, my ex and I broke up due to distance. I think I was pushing my feelings of uncertainty aside because I really did love him. If the relationship is stressing you out, it can be easy to believe it can get better, but if the issues you have already addressed are not changing, you are only hanging on for the comfort. I was sad in the beginning but I look back on the moments I felt exactly as you described and I have not looked back.