Mark Waser knows I DO NOT agree with the opinion that there is a universal right and wrong which any group of experts can figure out. Right and wrong is inherently subjective because the consequences are subjective. Right for Alice could be wrong for Bob. It's only when right for Alice is also right for Bob that they can agree something is right. But it doesn't mean Gordon will agree with either of them, because right for him could be wrong for them both.

So the point? We have competing interests, and while you might be able to try to determine what is right for all life on earth, it is doubtful you will be able to do this without knowing the preferences of all life on earth. No group of humans currently has the information processing capability to understand the preferences of all humans, let alone all lifeforms on earth, to be able to determine what is best for life on earth.

And then what is best for life in the universe? There could be life on other places which aren't earth which have different interests. Should our AI be only concerned with earth? Even if you create a Gaia AI, it doesn't mean much when you don't know anything about what else is out there.

There are no moral experts

And that includes me. I don't believe I can determine what is best for anyone. I don't even claim to know what is best for myself. I accept my ignorance, and understand my ignorance is based on my very limited human ability to process information. The only way to reduce my ignorance to make better decisions is to enhance my ability to decide. This is why decision support is in my opinion of greater importance right now than to recklessly try to create AI when we don't even have an understanding of the big data we collected. We should probably figure out how to reduce ignorance before trying to make sense of AI ethics, and this means a cautious human centric approach to AI until we truly understand the consequences of our actions.