This article from MIT Technology Review talks about how engineers don't fully understand how deep learning algorithms work their magic.
The legal world believes that we should be able to query this problem. If a deep learning algorithms is used to determine such life and death issues like who gets parole or not (or who benefits from health insurance or not), then we should know how that algorithm was able to make its determination.
The obvious answer to this conundrum is that engineers must design other algorithms, call them 'monitor algorithms' to study and decide these deep learning algorithms just like neuroscientists study the brain, but hopefully with better results!
