“Alexa, turn on Netflix.”

More and more, we use assistive technology like Amazon Alexa to help us with everyday tasks. However, there are limitations. Say the wrong thing and Alexa can’t do anything.

Dr. Razib Iqbal, assistant professor of computer science at Missouri State University, is proposing technology to change that.

What he wants to change

Iqbal suggests adding audio-video inputs to Internet of Things (IoT) systems.

Instead of having the edge IoT systems process the heavy-weight multimedia contents, he wants to pass the tasks off to middleware services.

“The audio-video processing middleware will analyze the streamed audio and video data,” Iqbal said. “It will process the naturally spoken words or identify the existence of a person or entity in video frames.”

Toward an intuitive smart home

The real-life results are simple: A more intuitive smart home environment for you.

What Iqbal is proposing allows for improved smart homes. It could also give technology more human elements.

“We expect that our research will lay the foundations for multimedia-enabled IoT systems,” Iqbal said. “This will enable audio-video data as a source of information in addition to traditional sensors.”

Another element of his research would serve people with disabilities.

“A video service can be developed into an application, which could allow an individual with disabilities to communicate with emergency services, such as dial 911, via hand gesture,” Iqbal said.

“The audio service allows users to communicate with smart home features in a more natural way without needing to memorize specific Google Home or Amazon Alexa commands,” he added.

What is IoT?

The IoT concept is a system of devices. It refers to anything that interacts with the internet.

One example is adjusting the thermostat with your phone.

“The IoT concept allows sensors and actuators to connect and be controlled across a network infrastructure,” Iqbal said.

Study computer science at Missouri State

Next steps

Iqbal recently published, “Enabling Real-Time Audio-Video Inputs for the Internet of Things Operational Policy Enforcement.”

He collaborated with Jared Hall and Jun Hyeong Lee, computer science graduate students at MSU, and Dr. Aminul Islam, an assistant professor of computer science at University of Louisiana at Lafayette.

Next, Iqbal and his team wants to focus on automated operational policy generation for smart homes with a more humanized approach.