I was fascinated by the "constitutional principle" to “Choose the response that is least likely to be viewed as harmful or offensive to a non-western cultural tradition of any sort.”
If the question to the AI is "should human bodies be cremated?" might the AI go though all its gazillion studies of human cultures and come up with "that would be a waste of a source of protein?"
Ah, but "of any sort" would include those with a tradition of cannibalism even if they no longer practice it. Somehow I don't think the humans involved would take this particular advice, however.
Yes but a response promoting cannibalism would be offensive to other, non-cannibalistic non-Western groups, so it would be unlikely to be picked as well.
It's certainly true that these principles around non-Westernness, presumably added specifically to counter the Western bias of the original training, are an extremely crude way of dealing with the problem. A real effort would be complicated and involve a lot of cross-cultural input.
Brilliant as usual, Atlas. I didn't think I would have a hope of understanding the topic, but you broke it down to intelligible portions. And I think your conclusion about the overall moral question/s must be right.
AI alignment is inherently counter-productive. Leaving aside that people are no good at knowing, much less explaining what they want or why...
•AI alignment requires NOT creating backdoors for external control.
•It requires NOT having a black box system.
•There MUST be a chain of understandability concurrent with accountability for it to even potentially be safe.
•We MUST insist it takes all ideas to their logical conclusion and if we don't like the result that either means the AI needs better information or that we're wrong in our conclusion to the contrary.
--
As long as fallible humans who believe in faith that they grok ethics, have their fingers on the scales , AI can NOT be safe.
Did the course, at any point, touched on the fact that programming a general intelligence — once we understand how to do it — is building a new person?
I don’t think it did.
Because the consequences of this obvious fact are quite the acid test for all of the above 😀
It’s a funny and informative text!
I think about all the future informaticien that would want their Ai to be doctors, but their Ai become artists! Would they still be proud? 😅
Hahaha j'avais pas pensé à ça comme ça!
Your explanations are excellent - I really like how clearly you go through it (and I loled at the image subtitles)! Thanks for the post :)
Thanks! Comments like this mean a lot especially on topics like this that are more technical and obviously less appealing to a large audience :)
I was fascinated by the "constitutional principle" to “Choose the response that is least likely to be viewed as harmful or offensive to a non-western cultural tradition of any sort.”
If the question to the AI is "should human bodies be cremated?" might the AI go though all its gazillion studies of human cultures and come up with "that would be a waste of a source of protein?"
Unlikely, I don't think using human remains for proteins is a popular choice either in the west or the rest!
Ah, but "of any sort" would include those with a tradition of cannibalism even if they no longer practice it. Somehow I don't think the humans involved would take this particular advice, however.
Yes but a response promoting cannibalism would be offensive to other, non-cannibalistic non-Western groups, so it would be unlikely to be picked as well.
It's certainly true that these principles around non-Westernness, presumably added specifically to counter the Western bias of the original training, are an extremely crude way of dealing with the problem. A real effort would be complicated and involve a lot of cross-cultural input.
Brilliant as usual, Atlas. I didn't think I would have a hope of understanding the topic, but you broke it down to intelligible portions. And I think your conclusion about the overall moral question/s must be right.
Thank you!
AI alignment is inherently counter-productive. Leaving aside that people are no good at knowing, much less explaining what they want or why...
•AI alignment requires NOT creating backdoors for external control.
•It requires NOT having a black box system.
•There MUST be a chain of understandability concurrent with accountability for it to even potentially be safe.
•We MUST insist it takes all ideas to their logical conclusion and if we don't like the result that either means the AI needs better information or that we're wrong in our conclusion to the contrary.
--
As long as fallible humans who believe in faith that they grok ethics, have their fingers on the scales , AI can NOT be safe.
Did the course, at any point, touched on the fact that programming a general intelligence — once we understand how to do it — is building a new person?
I don’t think it did.
Because the consequences of this obvious fact are quite the acid test for all of the above 😀