The world is still new to this Pokémon Go thing, Nintendo’s mobile game that has seen instant success since its release last week and sent players stumbling around in public with their smartphones out. But of course, there are bound to be things that go a bit funny in any game, like when Pokémon Go thinks your house is a gym for training all the virtual creatures people have captured.

A Massachusetts man who lives in a house that as built as a church in the 1800s started playing Pokémon Go on Friday. As soon as he got home, he noticed something was off: his house — which he and his wife had moved into a year ago — was showing up as a “gym” in the game, which is a spot where players can train and fight Pokémon.

“I thought, that can’t be right,” he told Buzzfeed News.

The next morning, people began arriving: mostly teens or young kids, and they were staying there and staring at their phones. Because unlike simply capturing a Pokémon on a landmark, to train them at a gym, you have to remain in the vicinity to work out your monster:

Players showed up throughout the night, which he said felt a little creepy. Some realized, however, that they could sit across the street and still be “at” the gym.

So why was his house playing host to all these players? He thinks it’s because many churches in the game are designated as gyms, and his house used to be one. He believes “no one bothered to check and see if it was an actual church or if it was a residence.”

In the meantime, he’s spent some time with the new arrivals, hoping to meet his neighbors. He hopes those neighbors aren’t upset by all the people showing up, however.

Despite the fun side of things, he says it’s a bit weird that he doesn’t have any control over this situation, and never signed off on having his residence included. Overall it doesn’t bug him, but the nighttime visitors aren’t the best.

“I’d be cool with it if I could have some control over the hours,” he told Buzzfeed News. “I’d rather them get it sorted out a little bit better.”

He’s now hoping to talk to the game’s developers to ask if they could put some limits on when people show up at his house, or move the gym across the street to the park.

Want more consumer news? Visit our parent organization, Consumer Reports, for the latest on scams, recalls, and other consumer issues.