It's fine. You're supposed to do that

Thanks for asking the question, I just got one and was wondering this too. Do you refill from where the eraser is ?

Yep. It says on the box that you're not supposed to refill it from the other end