While I agree that magic shouldn't be completely predictable I hate it when any sort of plot hooks, sci-fi doodads and supernatural elements are overused to suddenly change to story. It's like with a good detective story, you need to show the audience the truth, at least in some form, while hiding it just well enough to confuse us. Perfect examples are the Usual Suspects and Twelve Monkeys.
I haven't read Game of Thrones but it seems really cheap in this aspect, he just makes up magic or creates characters whenever he needs them, making me care less because there is no finality, the plot can change at any point just because of new characters.
Concerning the popularity of magic and sci-fi, it's mostly because publishers don't want to risk original IPs. It's easy to sell a shitty sci-fi flick with no resemblance to the source material if you market it to the fandom of the source. And to be frank almost all fantasy films I've seen are mediocre at best.
Probably the only good, traditional fantasy I’ve seen is Solomon Kane (as an honest fantasy film, I can't comment on the adaptation quality), other than that it’s fables (for example Pan's Labyrinth, Hero, Crouching Tiger, Hidden Dragon) or comedies (Princess Bride, Stardust, Hansel & Gretel, Mummy). With sci-fi films it’s often the case of slapping some poorly thought-out elements on a mundane story (Divergent, Next, Avatar, Host, Source Code) or TV shows drawing us with an interesting premise and then completely failing to deliver anything but a soap opera (Falling Skies, Revolution, Outcasts, Terra Nova, 100, Continuum, Defiance, Defying Gravity).
In all honesty, sometimes I wish fantasy and sci-fi wasn’t so popular among the publishers, that way we could avoid having lunatics like Jar Jar Abrams devour our childhood memories of old sci-fi, turn hardcore source material into PG-13 pop culture piece (Kick-Ass) or bastardize the source material in general, as it’s been for example with P.K. Dick’s works. Let’s hope it’s just a phase…