I personally think that all the criticism of them is bullshit, and that the people who campaign against them and cite them as ever are either religious fanatics or don't totally know what they're talking about.
I don't see how there's an ethical problem. Religious people might make one up about how it's not God's will, but I really don't see how breeding special new species of plants in a laboratory is in any way an ethical issue.
The only real issue with them is that genes may get transferred. You create a plant immune to weedkillers, so you can blast the fields with the stuff and it won't die. Unfortunately, somehow this gene gets transferred to weeds, and you get MRSA-style weeds in your fields, killing your crops.
See, this might sound plausible and reasonably scary, but in reality it's not. Cross-pollination is pretty damn rare when you're talking about different species, and the likelihood of any actual genes being passed down, let alone the modified one in question is negligible.
And even if they do get passed down, it doesn't put us in a situation much worse than if we didn't have the GM crops to begin with. It's a bit like banning the use of all antibiotics so that they don't all become completely useless. The cure is self-defeating.