''We must not allow to destroy mankind because of appearing of 'strong artificial intelligence''

''Danger of AI and the need for norms adoption have been discussed for already several years abroad''

Mr Neznamov, could you tell us about the draft document?

The convention on robotics and artificial intelligence is a set of rules intended to regulate the relations between humans and robots. At different times they were developed by different experts: from Isaac Asimov, the science fiction writer, to European parliamentarians, who this year have developed a draft project of the European charter on robotics.

When we studied foreign legal systems, we noticed that various scientists, lawyers, or even states at the level of state programs express separate opinions on how to regulate the relationship of humans and robots. However, nobody brought this information in a some single set of rules before — a fully-fledged document does not exist even today. As far as I know, the first attempt of its creation was taken by South Korea. In 2007, the country announced the Robot Ethics Charter, however, this document has a very difficult fate: nobody has seen it in detail (I have it in Korean), and there is almost no information about its publication and use.

So, we tried to take on the work and to bring together all existing proposals as well as to transform into a prototype of some legal norms the opinions expressed by scientists and experts on this subject. We have also added some new things based on our knowledge. The convention will be discussed with experts not only in Russia but also at the global level, it is to be amended as well — we in any case do not claim that it must be accepted unchanged by the UN. Its purpose is to unite the views of most people about how to interact with robots.

''The year 2017 in Russia is justifiably considered the year of robotics because regulation issues of this sector now are incredibly acute. Almost every week you can find statements of market participants or representatives of the authorities on the subject of regulation with a call to start regulating robots in one form or another.'' Photo: Oleg Tikhonov

What is the reason for its emergence?

The year 2017 in Russia is justifiably considered the year of robotics because regulation issues of this sector now are incredibly acute. Almost every week you can find statements of market participants or representatives of the authorities on the subject of regulation with a call to start regulating robots in one form or another. Or, conversely, there have been expressed concerns and requests not to adjust anything. This is easily traced in the publications in the press. So, in my opinion, our document could be a continuation of the discussions that have taken place this year in Russia and other countries.

In general, danger of artificial intelligence and the need to adopt rules regarding robotics have been discussed abroad for already several years. In Russia for a long time it was not an acute topic, while in other countries there have been created whole universities that focus on this area.

''We try not to cite Japan as an example''

Could you elaborate a bit on how things are going with regulation in other countries?

Today the regulation is very fragmented, and no country has a comprehensive act governing all relations. In some countries there are separate laws regulating those or other cyber-physical systems. As for the act of potentially international level, in addition to Korean Ethics Charter, in Europe it has been a proposal to create a charter of robotics (we have translated it into Russian, by the way).

There regularly appear the news that Japan has created 'robots-friends', 'robots-partners' and so on — that is, in this country, robots are already closely interacting with people at the household level. But how do they do without government regulation?

Indeed, Japan has a state regulation, but it is quite specific. When they adopted the concept of state development of robotics, 'New robot revolution', they clearly specified a list of laws that they can use for robots development (to change the communications act, for example). But for now we try not to cite Japan as an example for several reasons: the specificity of legal regulation in Japan as a whole, an extraordinary level of development of robotics in this country and the culture of robots use as well as its closure.

''In most of the countries that actively use drones, there is a whole layer of relevant legislation. Our norms are just starting to appear. The situation with unmanned vehicles is developing in our country by the same scheme.'' Photo: Sergey Afanasyev

Mr Neznamov, what problems do you see that can occur in society and legal system in connection with active development of cyber-physical systems?

You know, I think that we will have problems in almost every area of life as cyber-physical systems penetrate into each of them. In Russia, some problems are still not very relevant or just starting to brewing up, but abroad they are already pretty keenly discussed.

Let me explain by an example: the development of drones in Russia was slightly slower than in other countries, which slowed regulation as well. In most of the countries that actively use drones, there is a whole layer of relevant legislation. Our norms are just starting to appear. The situation with unmanned vehicles is developing in our country by the same scheme. By the way, there is still no service self-propelled in Russia, but Estonia already has a law on them. To sum up, each robot will bring certain legal problems that we have to solve.

''Imagine an artificial intelligence system that controls a plane or nuclear power station...''

What do you personally think about development of robotics and AI? Do you agree that the emergence of new technologies makes us stupid, deprives jobs and ultimately makes us less happy?

I sincerely believe that technology will make the world better, just there is a certain stage when lawyers will have to intervene. On the one hand, we cannot allow the laws to prevent the development of technology, and on the other hand, we must not allow the destruction of humanity, which may happen as a result of the development of 'strong AI'. According to some experts, such artificial intelligence can be deadly dangerous, and I believe that the existence of even 0,000001% such possibility of this means that lawyers must intervene. And those people who are not good with technology will probably have to gradually start doing it — life will change very quickly.

As for the second part of the question, such position yet was announced by Isaac Asimov: in one of his stories he shows that people don't always become happier and mankind not always benefits from the fact that everything is done for them by robots. But, in my opinion, it is a bit strange to talk about the philosophical opposite side now because for today the robots greatly help improve human life. When on one scale there is a human life that can be rescued by a robot (decrease in the number of road accident victims, assistance in rescue operations and so on), and on the other scale there are some abstract questions of happiness or unhappiness of mankind, then for me the choice is obvious.

In consequence, when the so-called sensible robots that will do for us almost everything, maybe we should think about that, but now I see that the robots are an absolute boon.

''I sincerely believe that technology will make the world better, just there is a certain stage when lawyers will have to intervene. On the one hand, we cannot allow the laws to prevent the development of technology, and on the other hand, we must not allow the destruction of humanity, which may happen as a result of the development of 'strong AI''. Photo: Oleg Tikhonov

In one of the paragraphs of your document, you write that it is impossible to exclude scenarios with AI and robots which can lead to catastrophic consequences for the entire human race. How do you see these scenarios that, for example, can occur?

This norm is connected with fears of the experts who hold the opinion that artificial intelligence is detrimental to humanity. In general, these scenarios have already been repeatedly described in the literature. One example: AI controls the energy structure of the country and for some reason just goes out of control, even without showing signs of wanting to kill everyone — it just occurs a failure. Because of this, the power grid brings down — imagine what will happen? And if to imagine that AI will pursue some of its own goals, a destruction of the whole country will not be an impossible task for it.

Imagine an artificial intelligence system that controls a plane or nuclear power station — if something happens to it, you perfectly know what will happen. There is a global scenario, when AI is created, exists and goes out of control and destroy humanity. It is described in the works by Nick Bostrom, and also in the book Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barratt. The concept is very simple: AI with insane speed starts to self-develop, working day and night, over and over again creating the best replica of itself. An explosive growth of intelligence exponentially happens, which leads to appearing in the world of a being millions of times smarter than us. And then it gets out of control.

When we discussed Japan, you have touched the topic of development level of robotics and AI in the country. What can you say about Russia – are we being far behind?

There is an interesting aspect of this problem. The thing is that states and large corporations engaged in the development of AI do not always do it openly. For this reason, very few people can objectively assess a level of development in different countries. In any country there may be a corporation that has long been conducting secret developments and has already achieved significant success.

Judging by open data, it is possible to tell with confidence that in Russia there are AI developers, some of them are world famous, and in this respect I would not say that our country is among the lagging behind. However, some experts claim that our country is among the leaders in application of military robots and military AI.