Hands Free Interface for Human Computer Interaction

At present various technologies are being developed for the betterment of differently-abled people worldwide. Through these, disabled people are given the opportunity to lead a normal life style like every other human being. But, still, disabled people don’t have proper facilities to use a computer conveniently, which in today’s context is a most basic activity done by all of us.

This problem has been comprehensively addressed through a research done by 4 final year undergraduates in University of Moratuwa, in 2010, namely, K.C. Samarawickrama, H.M.L. Chandima, K.G.T.D. Chathuranga and S.K. Chathuranga from the Electrical Engineering Department, under the supervision of Dr. Harsha Abeykoon.

This product provides an alternative to the conventional mouse using computer vision and voice recognition technologies. The user can use his or her face movements to move the mouse pointer and use other mouse actions by voice commands. Hands free interface consists of f two hardware components, a web camera and a microphone. The web camera is expected to be pointed to the face of the person who is operating the computer. The software component accesses the video stream coming from the web camera and moves the mouse pointer accordingly with the nose movements. The microphone captures the voice of the user and detects the voice commands which are used to issue mouse actions such as click, double click and right click. Drag and Scroll events also can be done.

Two main Technics used in implementing Hands Free Interface for Human Computer Interaction are image manipulation based nose tracking and speech recognizing based mouse command issuing part.

Image manipulation was done through face detection and nose tracking. The face detection is first done to identify the nose and only then the tracking is done. Nose tracking plays an important role in the entire system, so a robust and effective tracking algorithm is essential in designing the system.

In Speech Recognition and Mouse Event Detection, Microsoft SAPI (Speech API) was used as the voice command detection engine. SAPI provides functionalities for recognizing and synthesizing human voice.

Several tests have been conducted to upgrade the quality of the product like ‘success probability test for different noise levels’, ‘Awkwardness of Using “Nose Mouse” with the Distance between User and Screen’ and ‘Initial Nose Identifying Time’.

‘Hands Free Interface for Human Computer Interaction’ becomes handy in enhancing computer literacy among differently-abled children and many more instances. Thus it does a greater contribution to enlighten the technological knowledge of disabled people which would definitely help them in keeping up the pace with the rapidly developing world.


University of Moratuwa, a leading technological university in the region welcomes you to witness a truly unique experience!

Contact Us

  • Tel: +94 112650301 +94 112650188
  • Fax: +94112650622
  • Email: info AT mrt.ac.lk