Then maybe they shouldn't have modeled it after the magnet. Instead, it should have been Mr. Telepathic Dwarf, who attracts certain objects and not others, and it's pretty arbitrary which objects are and aren't attracted. After all, the grinning key doesn't exactly look like metal either.
That would have made just as much sense. It's the Isle of Wonder after all!