|
The main legal-ethical issues raised by SUBITO
are around privacy. A privacy impact assessment was carried out
using the general theoretical and practical aspects of the social
and legal impact assessment. Where possible, input has been provided
to the technical development undertaken in WP400 and WP500 activities.
However, there was a significant time lag between the project's
original Social, Legal and Ethical (SLE) expert leaving the UK and
new SLE becoming a full project partner. There has therefore been
a less-than-ideal linkage between the Privacy Impact Assessment
(PIA) and technical development processes.
Research was also carried out into the social,
legal and ethical requirements related to automated surveillance
monitoring of public spaces and identification of individuals.
This included an assessment of approaches which are either illegal (
or may become so) or which may be unacceptable to the public.
Building on the Privacy Impact Assessment and the report produced
by Leeds University on the ethics of automated threat detection
this research is documented in deliverable 300.4.
The ethical impacts of the approach to threat
assessment developed within SUBITO was investigated. To place
this in context, a range of different threat assessment systems
were considered. Manual assessment was seen to be ethically flawed
in a number of ways, not least because of individual prejudices
and natural limitations of the human operator. As such, automated
threat assessment offers a promising alternative as a means of
eliminating the human element to overcome these flaws and eradicate
prejudice. However it was seen that prejudice can still enter an
automated system and, through its presence in the underlying code
will become institutionalised. A second suggestion of automation
was the eradication of false positives, something which was seen
to be true for the system, but not the end user. Similarly, the
concerns of function creep and distance between surveillant and
surveilled were seen to at least continue if not exacerbate under
automation.
While automated threat assessment does have some
ethical advantages over its manual alternative, then, these are
counter-balanced by competing problems introduced by the nature
of automation. SUBITO, though, is neither manual nor fully automated,
but a partially-automated or "advisory" threat assessment tool.
It is designed to aid operators in dealing with information overload
and recognizing threats which may be missed through operator error.
This it achieves by sorting people on the behavioural criteria of
leaving baggage unattended. By specifying the precise nature of
the threat SUBITO both limits its applicability and its ambiguity
in a way that defining threats as "suspicious behaviour" does not.
Having identified targets SUBITO applies two further criteria of
analysing behaviour in crowds and goal prediction to filter out
false positives. These are reduced still further by continued
human input ensuring that operators and ground staff are involved
in the situation, a factor which also helps to counter dehumanization.
That said, there are still ethical concerns regarding
SUBITO, not least the possibility of institutionalizing prejudice
through the use of the way in which group relationships and intentions
are inferred. In both cases the analysis is based on data sets
drawn from European contexts and so may prove to demonstrate a
bias in favour of European subjects. This could lead to a number
of false positives from a similar socio-ethnic background, leading
to their being treated unequally. It might also reduce operator
trust in the system, and so to an increase in false negatives,
if the alerts always concern similar socio-ethnic groups. Overall,
though, the analysis within the SUBITO system serves to reduce the
number of false positives.
There is little obvious scope for function creep
or abuse of SUBITO as a system. The parts of SUBITO and particularly
the potential of the threat analysis to identify minority groups
or undesirables, or to instil chilling effects on a population,
do have potential for function creep. It is therefore important
to recognise that the ethical concerns related to the project go
beyond the finished product itself. There is also a concern regarding
function creep and automated threat analysis as a concept. As
automation is trialled and found to be successful in limited contexts,
so it will become more popular and more developed. Unless ethical
evaluations are provided at every stage of development, what is
seen as acceptable at this level of automation and in this context
might not remain so as the technology is taken in new directions.
It is therefore essential that this is not seen as the final word
on the ethics of SUBITO or on automated assessment in general,
but as part of an ongoing dialogue.
|
