THE JOHNS HOPKINS UNIVERSITY
OFFICE OF COMMUNICATIONS
3910 Keswick Rd., Suite N-2600
Baltimore, MD 21211
Phone: 443-997-9009 / Fax: 443-997-1006
October 31, 2017
MEDIA CONTACT: Phil Sneiderman
Office: 443-997-9907/Cell: 410-299-7460
[email protected] On Twitter @filroy
Newswise — Imagine an embassy bombing. Consider the massive amount and varied types of data that investigators would need to review to determine who carried out the attack and how it was done. Such a probe could involve the slow, painstaking examinations of video footage, photos, internet communications, telephone records and other material.
A Johns Hopkins University-led international team of scientists, supported by an $11-million, five-year U.S. Defense Department grant, wants to streamline such investigations by developing algorithms for extracting the most useful information from multi-modal data. Participating scientists from nine universities in the United States and the United Kingdom will convene at Johns Hopkins Homewood on Wednesday, Nov. 1, for their first group meeting on the challenging project.
The team’s ultimate goal is to teach a computer system to “think” like a digital Sherlock Holmes. It would quickly identify the most useful information in the data and ignore details it deems irrelevant.
René Vidal, a Johns Hopkins biomedical engineering professor who is principal investigator on the DOD grant, said a key goal is to develop the technology that will enable a computer to characterize the information content of multi-modal data. Taking images as an example, the goal is to develop methods that can distinguish what is happening in a particular photo and not just store it as another JPEG file.
“In a computer today,” Vidal said, “a picture of a car and a picture of a person are compressed in the same way. We want the computer to recognize what objects are present in a photo or video, what actions are taken, and to see what the contextual relationships among these entities are. That’s what we call the semantic constraints of the scene.”
For example, if a photo or video depicts a large truck parked behind a grocery store, where workers unload fresh fruit and vegetables, then a computer programmed to look for unusual or potentially deadly activity would likely dismiss that scene as mundane business as usual. But if the same truck approached a large group of protesters or people gathered for a celebration, the system would signal an alarm, based on the recent series of terrorist truck attacks on pedestrians.
“The system is going to be task-dependent,” Vidal said. “If someone tells me to count the number of times a truck comes to a supermarket to unload fruits and vegetables, then a picture of that happening is very important. But if the task is to try to detect a truck that might attack people, then the truck unloading fruit at the supermarket will be considered irrelevant.”
He added that the more intelligent computer system could provide invaluable time-saving help to the armed forces. “If you were in the military, and you’ve recorded 10 million conversations to find information about the enemy,” he said, “you’re not going to listen to 10 million conversations to find out what you need to know. That’s where the question of ‘which of those 10 million conversations contains something informative’ comes in. And not only which conversation, but which part of the conversation has something important in it?”
Vidal pointed out that other applications of intelligent computing advances are already turning up in pilot projects such as self-driving cars and the Amazon Go retail store that allows shoppers to fill their carts and depart without going through a conventional checkout cashier. Instead, the merchandise in the cart is tracked by the store’s computer system and is automatically charged to the customer’s Amazon account at the exit.
The funding for Vidal’s team was awarded this year through the DOD’s Multidisciplinary University Research Initiative program. In addition to Johns Hopkins, the other U.S. schools participating in the project are Stanford University, the University of Maryland, UC Berkeley, USC and UCLA. The U.K. researchers are from Oxford University, Imperial College, University of Surrey and University College London.
At Johns Hopkins, the Department of Biomedical Engineering, in which Vidal holds his faculty appointment, is shared by the university’s Whiting School of Engineering and its School of Medicine. Vidal also is a core faculty member within the university’s Center for Imaging Science.
Color photo of Professor Vidal available; contact Phil Sneiderman.
###
To arrange a video or audio interview with a Johns Hopkins expert, contact a media representative listed above or visit our studio web page. Find more Johns Hopkins stories on the Hub.
RSS