Because if its a why not question, its not science. If it doesn't fit the scientific method, its not science. It is, however, experimentation, which doesn't need a hypothesis.
Why not is not the question for science. The abstract science question in that case would be "How far can we go? With what can we get through? How would a Turing test go with this one?"
But it was not real intelligence... Anyways, SWEET!
It could successfully replace many other people who think that they are intelligent. So at least it is a worthy artificial self-deception.
The true mastery of AI would still be having a computer that is able to lie in a controlled way, as you need for proper strategy. I have yet never seen any AI that is capable of deceiving a human player, without having a human teaching it deception patterns in advance.
You're right, there is no difference. Both are man-made
Not really that... but rather that what we call intelligent, can be with different measures be simply stupid. And people we quickly mark as stupid, can often be actually very intelligent, but just not fitting into the scheme of intelligent people, as such people see themselves. It is very easy to make stupid errors, if you are too much used to defend stupidity as ingenuity.
As if you make your kid more intelligent if you only permit violin or piano as instruments, despite electric guitar or drums actually requiring a lot more mastery if you want to play in one league with the best. Classic music is actually one of the most uninspired fields of music, that only enforce stupid learning of standard patterns. Just like Jazz, but good Jazz at least tries to escape its curse. And if you would have the nerves to let your kid learn playing bag pipes, you would at least force it to learn something that requires a few months of training before you can actually try playing the first tunes on a real bag pipe.
Maybe that is another good way to define intelligence: Being able to think out of the box. A poor AI can maybe learn, but not learn to go beyond what its programmer taught it in advance. A good AI would not limit itself to its program, but rather "invent" new rules for it, reorganize itself and adapt to become better, and also learn to try something that is not directly leading to the result for the sake of having more options.
But this would also mean that we would have to accept failed AIs, which drew the wrong conclusions and optimized themselves to the wrong world... despite having worked properly for decades.
