Stroke patient may be overwhelmed by visits

DEAR DOCTORS: A man I have been dating had a stroke a few weeks ago, and now he has aphasia. When I visit him at the rehab, he's ready for me to leave after a short time. Is this normal? He seems to be happy I'm there, but he dismisses me quickly. I'm trying not to feel hurt.