My worry is that we may be forced to empower A.I. systems with more and more decision making capabilities based on their algorithms (due to the complexity of various tasks) and that eventually they may become too autonomous. Maybe they might become so powerful that they start to disagree with us about certain matters to the detriment of us.
The movie "The Matrix" seems instructive here. Ok, so it was mostly a cheeseball sci-fi cops and robbers show, kinda disappointing that, but still, a great concept.
Will we care if AI takes over if they provide our brains with a compelling fantasy reality where our egos get whatever they most want?
Here's an example.
This forum is a form of virtual reality. In the real world very few people want to discuss these topics to the degree we nerds do. We respond to this by abandoning the real world to some degree and investing some, or maybe much, of our time in an entirely abstract realm where we can get what we want.
We happily give up names, faces, sights, smells, voice and most aspects of a real life human experience so that we can get the kind of stimulation we desire in this quasi-virtual realm.
Someday somebody will enhance social media with the creation of fully virtual non-human posters. We'll be able to customize these virtual entities to deliver whatever kind of experience we most desire. Want to endlessly discuss your favorite nerd topics with gorgeous college co-eds who think you are the most brilliant guy they've ever met? Just tweak some dials in your forum control panel, and they're yours.
The first generation of humans to encounter this technology will say, "oh no, no, no, I'm not talking to them, they're not real!" But succeeding generations born in to this world will accept virtual entities willingly and without complaint, just as today's youth take naturally to the Net.
Here's primitive example of talking to a virtual reality entity available on the net today...
And so the question arises...
Why would you waste time talking to me, a real life human geezer blowhard dude, when you could be talking to a room full of virtual gorgeous college co-eds who think you're REALLY hot?? I assure you, I don't think you're really hot, sorry, I just don't. :-)
Point of rant being, AI may imprison us in the Matrix, and we may be very happy to accept that fate.
If you think about the history of human culture, once we've met the needs of the body, we very often become entranced by fantasy realities of various kinds, art, religion etc. Think of ancient cavemen sitting around the fire telling each other stories. Once they had food in their bellies, the stories tended to become a focus.